Sebastiaan de With, on the Halide blog, goes deep on the 2020 iPad Pro’s camera module. His examination reveals that the device’s wide camera is virtually identical to that of the 2018 model. And the ultra-wide camera, unfortunately, isn’t quite up to the quality level of what’s found in the iPhone 11 and 11 Pro.
The most exciting and impressive aspect of the camera system is the LiDAR Scanner. The Halide team actually went to the trouble of building an entire proof of concept app that utilizes the LiDAR Scanner to capture your surroundings.
With Halide, we’d love to use the depth data in interesting ways, even if it’s low resolution. There was only one problem: there are no APIs for us as developers to use to get access to the underlying depth data. They only expose the processed 3D surface.
What if we re-thought photographic capture, though? We built a proof-of-concept we’re calling Esper.
Esper experiments with realtime 3D capture using the cameras and LIDAR sensor at room scale. It’s a fun and useful way to capture a space.
I always love reading de With’s in-depth explanations and comparisons of new iPhone or iPad cameras, and this was an especially fun one.
It was the summer of 2016 when Pokémon GO took the world by storm, and now, almost exactly three years to the day later, there’s a new AR game trying to capture the same magic. Harry Potter: Wizards Unite launched today in the US, UK, and select other countries from the same studio behind Pokémon GO, Niantic.
At the core of Wizards Unite is the same AR-based system employed in Pokémon GO, whereby you can explore a digital world that’s mapped to the real world around you, where different real-life locations are mapped to in-game hot spots for engaging with certain game elements. Special locations in the game, such as fortresses, are designed for teams of players to conquer together; combined with the AR world exploration mechanic, this encourages strong community engagement in the game. If Wizards Unite takes off in any way like Pokémon GO, expect to see bands of players roaming the streets with their phones at the ready.
Chronologically, Wizards Unite takes place following the original seven book Harry Potter series but still includes many of those original characters. In the game’s tutorial, for example, a slightly aged Harry Potter, now working as an auror, instructs you regarding a crisis the Ministry of Magic is currently facing that needs your assistance. This sort of tight integration with the world and characters Harry Potter fans know and love, combined with the community-centric element found in games like Pokémon GO, could make Wizards Unite a really special experience for anyone who loves J.K. Rowling’s wizarding world. As one of those people myself, even though I never got into Pokémon GO, I’m excited to spend some quality time trying out Wizards Unite.
You can download Harry Potter: Wizards Unite on the App Store now.
Edgar Alvarez reports for Engadget on an upcoming AR feature I can’t wait to try:
Nike has been experimenting heavily with augmented reality for a few years now, and the company is continuing to work on new experiences powered by the technology. The sportswear giant is now introducing Nike Fit, a feature that uses a combination of computer vision, scientific data, artificial intelligence and recommendation algorithms to scan your feet and find the right shoe fit for you. And you can do it all in augmented reality, using the Nike app on your smartphone. Nike says that, according to industry research, over 60 percent of people wear the wrong size shoes. With Nike Fit, the company is hoping to solve that problem.
The AR experience itself is fairly simple: You open up the Nike app, go to a product page and, next to where there’s usually a menu that lets you pick the size of your shoes, you’ll see a new option to measure your feet. From there, the camera will pop up and you’ll be asked to stand next to a wall and point your smartphone at your feet, which will prompt a view that uses two AR circles to level your phone. Once the feature recognizes your feet and your physical environment, it starts scanning your feet and then tells you your ideal shoe size for Nike footwear. The entire process takes less than a minute.
It’s been two years since Apple introduced ARKit, yet there are still very few AR apps I’ve found add meaningful value to my life. Outside of measuring apps, the only ARKit experiences I’ve enjoyed related to shopping for furniture and other home accessories. It looks like shoe shopping will be the next major area where AR becomes more than a fun demo. According to Alvarez, the Nike Fit feature will be added to the Nike app this July.
Later this year the team at Niantic will launch a new game based on a very popular IP: Harry Potter. The game’s full title is Harry Potter: Wizards Unite, and it will come to both iOS and Android as an AR-centered gaming experience in the vein of Pokémon GO, Niantic’s biggest hit.
Today Nick Statt of The Verge published his impressions of Wizards Unite following a preview Niantic hosted recently. Statt writes:
The core activity in Wizards Unite involves collecting a mix of artifacts and sometimes saving notable characters from the series who are stuck in a bad situation, like the titular Harry being pinned down by a spooky dementor. From there, you play a little tracing mini-game, to evoke the casting of a spell and then collect the item or free your friend. Afterward, you’re able to add that item or character to your collections book, earn experience points, and level up.
Beyond traversing the map, collecting those artifacts, and visiting inns to eat food, players of Wizards Unite will have a few more advanced activities to keep them busy. Those include leveling up your character, picking a subclass (called a “profession”) to learn new abilities, and then teaming up with up to five other players to compete in a fortress (what Niantic has designed to be this game’s version of gym battles from Pokémon Go). These team challenges feel like a cross between a traditional strategy game and something similar to Nintendo’s Fire Emblem, where one-on-one battles take place simultaneously after players select an enemy from a top-down map.
Based on Statt’s article, Harry Potter: Wizards Unite is going to have a lot more depth and complexity to it than Pokémon GO, which I’m excited about. Pokémon GO has scaled over time to become more of a gamer’s game than it used to be, but with Wizards Unite it sounds like that added depth will be there from the start for those who want it.
There’s still no firm release date available for Wizards Unite, but a 2019 launch is confirmed.
Last week, The New York Times announced that it had added an augmented reality feature to its iOS app. The first article with embedded AR content was a preview of the feature published last week that explained to readers how it worked. At the bottom of the article was a newspaper box that could be dropped into your surroundings. I showed it off to some friends over the weekend, and everyone was impressed by how realistic it looked as they walked around the box in a neighbor’s kitchen.
This week, the Times rolled the feature out as part of its Winter Olympics coverage. In Four of the World’s Best Olympians, as You’ve Never Seen Them Before, the publication spotlights figure skater Nathan Chen, speed skater J.R. Celski, hockey player Alex Rigsby, and snowboarder Anna Gasser. The results are impressive. I placed each athlete in my living room, then walked around them. From each angle, snippets of text about what I was seeing were overlaid on the image providing additional details and context. The app also makes use of haptic feedback on the iPhone to alert users to new information as they examine a scene.
With the ubiquity of the smartphone in increasingly younger classrooms, integrating them into education is fast becoming a requirement for teachers. What better way to tame smartphones in the classroom than to make them part of the learning process?
Matthew Braun, developer of SketchParty TV (one of my favorite Apple TV games), released a new app to do just that. Waypoint EDU uses AR to make the phone not only a learning tool, but a truly interactive experience that can take place outdoors. Or anywhere.
From the student perspective, it works like this: students see a waypoint on a map of their current location and move around to find it. A la Pokemon Go, they search by looking through their phones, scanning for an out-of-place object such as a (miniature) colosseum sitting in a park. Once they’ve found the waypoint, they answer a quiz question to reveal the next waypoint. Think augmented reality geocaching.
From the teacher (or parent) side, creating a curriculum is pretty easy. I didn’t get into creating a fully customized one while I was trying it out, but editing the waypoints and related questions is simple. Once you have your curriculum set up, you just pull up the map and draw the playfield with your finger. The waypoints are automatically placed within the playfield, ready for the Hunt to be shared with the students via AirDrop. You can currently add artwork from a library, and the ability to add your own artwork will be a paid feature in a future update.
Waypoint EDU is a free app. Obviously, it has the requirement that everyone in the group has access to an iPhone. In situations where that’s possible, Waypoint EDU seems to me like the future of field trips. Below is a video of Waypoint EDU in action. You can find it on the App Store, and get more info at waypointedu.com.
In an update released for its iOS shopping app, Amazon has introduced a new way of viewing items from the online retailer: AR View. Built on Apple’s ARKit technology in iOS 11, AR View provides shoppers with a better understanding of how products will look when placed inside their homes.
AR View is accessed inside the Amazon app by tapping the camera button, then selecting AR View from the assortment of camera options. You’ll then get to browse through a limited selection of product categories, such as Living Room, Kitchen, and Electronics; there’s also a Top Picks section. Unlike the similar AR experience from IKEA Place, only one product can be previewed in Amazon’s AR View at a time. After placing a product in AR, you can move its position or rotate it, and pressing the button with three dots will take you to the full product page for initiating a purchase.
Amazon claims that thousands of items are available in AR View, but currently only a fraction of that estimate appears for me inside the app; we should except the number of AR-compatible items to grow over the coming holiday shopping season. It also wouldn’t be surprising to see AR View roll out to other parts of the app in the future, such that if you’re viewing the product page for an Amazon Echo, for example, there will be a button that allows you to instantly view the item in AR.
Today’s version of AR View is a first step toward enhancing the Amazon shopping experience with AR. There’s plenty more work to be done, but it’s exciting to see a glimpse into how transformative AR can potentially be for online shopping.
ARKit is one of the iOS 11 features I’m really excited about along with iPad improvements, changes to Notes, and better screenshot workflows. The “problem” with ARKit is that Apple isn’t offering a proper AR app for iOS yet – it’s a framework for developers to create AR experiences. Thus, until you play with an ARKit demo, it’s hard to understand the extent of Apple’s efforts and the potential for future ARKit-enabled apps.
Fortunately, the folks at Made with ARKit have been collecting early demoes shared by developers showing a variety of AR apps that will be possible later this year. And some of these are already incredible. From a rocket landing in your backyard to robots dancing in your living room and obvious measuring tapes, these videos give us an early glimpse at the promise of ARKit and the quality of tracking and rendering on an iPhone’s screen.
I don’t know if these showcases will turn into actual shipping products this Fall, but I have a feeling this new category of apps will become a great reason for millions of users to upgrade to iOS 11 quickly. I can’t wait to play with some of these AR apps.
At WWDC, Craig Federighi demoed ARKit, Apple’s new augmented reality API and mentioned that Apple was teaming up with IKEA on AR. The collaboration was mentioned again recently by Tim Cook in an interview with Bloomberg Businessweek.
Now in an interview with Digital.di, Michael Valdsgaard, Digital Transformation Manager for Ikea’s parent company, has provided further details of its upcoming AR app:
This will be the first augmented reality app that allows you to make reliable buying decisions.
IKEA has big plans for the app:
At launch, 500-600 products will be in the app. In future, it will play a key role in new product lines.
According to Valdsgaard,
When we launch new products, they will come first in the AR app.
Based on the interview, IKEA and Apple feel like a natural fit. IKEA has hundreds of 3D models ready to use with ARKit and Apple has a huge install base of iOS users for IKEA’s app. Moreover, IKEA can help demonstrate the types of applications that AR can enable beyond the gaming industry.