This Week's Sponsor:

Kolide

Ensures that if a device isn’t secure it can’t access your apps.  It’s Device Trust for Okta.

Posts tagged with "AR"

Beautiful Things for Spatial Computing→

I came across this fun website while browsing the Vision Pro community on Reddit: beautifulthings.xyz is a curated collection of 3D USDZ files that you can download for free on any Apple device. On the Vision Pro, these models can be freely placed anywhere in your environment alongside other windows, allowing you to inspect up close, say, a Spider-Man model, a Lamborghini, or, should you feel like it, a first-gen iPod classic.

Here is, for instance, a screenshot of a nice-looking Italian pizza and a Hylian shield just floating around my living room:

Pizza and Zelda? Yep, that's me.

Pizza and Zelda? Yep, that’s me.

According to the website’s creator, more than 100,000 items have been uploaded to the site in the past week alone, and the developers are working on a curated daily feed to showcase the best objects you can view on a Vision Pro.

Fun project, well worth a few minutes of your time even just for opening 3D models of stuff you can’t afford in real life. I hope they’ll consider adding a search functionality next.

Permalink

The Trouble with Mixing Realities→

Mark Gurman recently reported that Apple’s much-rumored headset will combine AR and VR technologies, which Brendon Bigley argues could be the wrong approach:

… I don’t think the road to mass adoption of virtual reality actually starts with virtual reality, it starts instead with augmented reality — a technology that can quickly prove its function if presented in a frictionless way. While even the best VR headsets demand isolation and escapism, a hypothetical product focused first and foremost on augmented reality would be all about enhancing the world around you rather than hiding from it.

Brendon’s story nails something that has been nagging me about recent headset rumors. The iPhone was a hit because it took things we already did at a desk with a computer and put them on a device we could take with us everywhere we go, expanding the contexts where those activities could be done. As Brendon observes, the Apple Watch did something similar with notifications. AR feels like something that fits in the same category – an enhancement of things we already do – while VR is inherently limiting, shutting you off from the physical world.

Like Brendon, it’s not that I’m not excited about the prospect of an Apple headset or the long-term prospects for virtual reality as a technology, but given where the technology is today, it does seem as though jumping into VR alongside AR could muddy the waters for both technologies. Of course, we’re all still working off of speculation and rumors. I have so many questions still and can’t wait to see what Apple has in store for us, hopefully later this year.

Permalink

Kara Swisher Interviews Apple CEO Cook for Sway→

Apple CEO Tim Cook was interviewed on Sway, The New York Times’ podcast hosted by Kara Swisher, in an episode released today. Swisher asked Cook about a wide range of topics, including privacy, iOS 14.5, Parler’s removal from the App Store, autonomous vehicles, AR, its upcoming court case with Epic Games, and more.

On privacy and the reaction to App Tracking Transparency, Cook said he was shocked by the degree of pushback the feature has caused. Asked whether he thought ATT will harm businesses that rely on digital advertising, Cook said:

I think that you can do digital advertising and make money from digital advertising without tracking people when they don’t know they’re being tracked. And I think time will prove that out. I’ve heard this about other things we’ve done in the past that it’s almost existential and it wasn’t. I don’t buy that.

Regarding Parler’s removal from the App Store, Cook explained that can return to the App Store when they comply with its rules:

Well, in some ways, it was a straightforward decision, because they were not adhering to the guidelines of the App Store. You can’t be inciting violence or allow people to incite violence. You can’t allow hate speech and so forth. And they had moved from moderating to not being able to moderate. But we gave them a chance to cure that. And they were unable to do that or didn’t do that. And so we had to pull them off. Now, having said that, Kara, I hope that they come back on. Because we work hard to get people on the store, not to keep people off the store. And so, I’m hoping that they put in the moderation that’s required to be on the store and come back, because I think having more social networks out there is better than having less.

Cook also made the case that human curation on the App Store is a crucial element of the marketplace’s security model, rejecting the notion that users should be able to sideload apps and elaborating:

I think curation is important as a part of the App Store. In any given week, 100,000 applications come into the app review. 40,000 of them are rejected. Most of them are rejected because they don’t work or don’t work like they say that they work. You can imagine if curation went away, what would occur to the App Store in a very short amount of time.

Regarding new products, Cook wouldn’t confirm whether Apple is planning to offer augmented reality hardware or an autonomous car. Still, his excitement about those underlying technologies was evident, noting that AR, in particular, is critical to Apple’s future.

Also of note was Cook’s comment that iOS 14.5 is ‘just a few weeks’ away, which is longer than I expected and perhaps a sign that an April product event will occur.

The interview, which is just under 36 minutes long, touches on other topics, including Apple’s role in policy issues like voting rights, working with the US government, and Cook’s role as the first openly gay CEO of a Fortune 500 company. The episode is available in Apple Podcasts as well as third-party podcast players, and The New York Times has published a transcript of the entire interview.

Permalink

Halide Team Experiments with iPad Pro’s LiDAR Scanner→

Source: Halide Blog

Source: Halide Blog

Sebastiaan de With, on the Halide blog, goes deep on the 2020 iPad Pro’s camera module. His examination reveals that the device’s wide camera is virtually identical to that of the 2018 model. And the ultra-wide camera, unfortunately, isn’t quite up to the quality level of what’s found in the iPhone 11 and 11 Pro.

The most exciting and impressive aspect of the camera system is the LiDAR Scanner. The Halide team actually went to the trouble of building an entire proof of concept app that utilizes the LiDAR Scanner to capture your surroundings.

With Halide, we’d love to use the depth data in interesting ways, even if it’s low resolution. There was only one problem: there are no APIs for us as developers to use to get access to the underlying depth data. They only expose the processed 3D surface.

What if we re-thought photographic capture, though? We built a proof-of-concept we’re calling Esper.

Esper experiments with realtime 3D capture using the cameras and LIDAR sensor at room scale. It’s a fun and useful way to capture a space.

I always love reading de With’s in-depth explanations and comparisons of new iPhone or iPad cameras, and this was an especially fun one.

Permalink

Harry Potter: Wizards Unite AR Game Launches on iOS

It was the summer of 2016 when Pokémon GO took the world by storm, and now, almost exactly three years to the day later, there’s a new AR game trying to capture the same magic. Harry Potter: Wizards Unite launched today in the US, UK, and select other countries from the same studio behind Pokémon GO, Niantic.

At the core of Wizards Unite is the same AR-based system employed in Pokémon GO, whereby you can explore a digital world that’s mapped to the real world around you, where different real-life locations are mapped to in-game hot spots for engaging with certain game elements. Special locations in the game, such as fortresses, are designed for teams of players to conquer together; combined with the AR world exploration mechanic, this encourages strong community engagement in the game. If Wizards Unite takes off in any way like Pokémon GO, expect to see bands of players roaming the streets with their phones at the ready.

Chronologically, Wizards Unite takes place following the original seven book Harry Potter series but still includes many of those original characters. In the game’s tutorial, for example, a slightly aged Harry Potter, now working as an auror, instructs you regarding a crisis the Ministry of Magic is currently facing that needs your assistance. This sort of tight integration with the world and characters Harry Potter fans know and love, combined with the community-centric element found in games like Pokémon GO, could make Wizards Unite a really special experience for anyone who loves J.K. Rowling’s wizarding world. As one of those people myself, even though I never got into Pokémon GO, I’m excited to spend some quality time trying out Wizards Unite.

You can download Harry Potter: Wizards Unite on the App Store now.

Nike Introducing AR Shoe Sizing Feature This Summer→

Edgar Alvarez reports for Engadget on an upcoming AR feature I can’t wait to try:

Nike has been experimenting heavily with augmented reality for a few years now, and the company is continuing to work on new experiences powered by the technology. The sportswear giant is now introducing Nike Fit, a feature that uses a combination of computer vision, scientific data, artificial intelligence and recommendation algorithms to scan your feet and find the right shoe fit for you. And you can do it all in augmented reality, using the Nike app on your smartphone. Nike says that, according to industry research, over 60 percent of people wear the wrong size shoes. With Nike Fit, the company is hoping to solve that problem.

The AR experience itself is fairly simple: You open up the Nike app, go to a product page and, next to where there’s usually a menu that lets you pick the size of your shoes, you’ll see a new option to measure your feet. From there, the camera will pop up and you’ll be asked to stand next to a wall and point your smartphone at your feet, which will prompt a view that uses two AR circles to level your phone. Once the feature recognizes your feet and your physical environment, it starts scanning your feet and then tells you your ideal shoe size for Nike footwear. The entire process takes less than a minute.

It’s been two years since Apple introduced ARKit, yet there are still very few AR apps I’ve found add meaningful value to my life. Outside of measuring apps, the only ARKit experiences I’ve enjoyed related to shopping for furniture and other home accessories. It looks like shoe shopping will be the next major area where AR becomes more than a fun demo. According to Alvarez, the Nike Fit feature will be added to the Nike app this July.

Permalink

Niantic Previews Harry Potter: Wizards Unite, the Studio’s Follow-Up to Pokémon GO→

Later this year the team at Niantic will launch a new game based on a very popular IP: Harry Potter. The game’s full title is Harry Potter: Wizards Unite, and it will come to both iOS and Android as an AR-centered gaming experience in the vein of Pokémon GO, Niantic’s biggest hit.

Today Nick Statt of The Verge published his impressions of Wizards Unite following a preview Niantic hosted recently. Statt writes:

The core activity in Wizards Unite involves collecting a mix of artifacts and sometimes saving notable characters from the series who are stuck in a bad situation, like the titular Harry being pinned down by a spooky dementor. From there, you play a little tracing mini-game, to evoke the casting of a spell and then collect the item or free your friend. Afterward, you’re able to add that item or character to your collections book, earn experience points, and level up.
[…]
Beyond traversing the map, collecting those artifacts, and visiting inns to eat food, players of Wizards Unite will have a few more advanced activities to keep them busy. Those include leveling up your character, picking a subclass (called a “profession”) to learn new abilities, and then teaming up with up to five other players to compete in a fortress (what Niantic has designed to be this game’s version of gym battles from Pokémon Go). These team challenges feel like a cross between a traditional strategy game and something similar to Nintendo’s Fire Emblem, where one-on-one battles take place simultaneously after players select an enemy from a top-down map.

Based on Statt’s article, Harry Potter: Wizards Unite is going to have a lot more depth and complexity to it than Pokémon GO, which I’m excited about. Pokémon GO has scaled over time to become more of a gamer’s game than it used to be, but with Wizards Unite it sounds like that added depth will be there from the start for those who want it.

There’s still no firm release date available for Wizards Unite, but a 2019 launch is confirmed.

Permalink

Media Outlets Use ARKit to Bring the News into Your Living Room

Last week, The New York Times announced that it had added an augmented reality feature to its iOS app. The first article with embedded AR content was a preview of the feature published last week that explained to readers how it worked. At the bottom of the article was a newspaper box that could be dropped into your surroundings. I showed it off to some friends over the weekend, and everyone was impressed by how realistic it looked as they walked around the box in a neighbor’s kitchen.

The New York Times debuted its new ARKit features with a newspaper box demo.

The New York Times debuted its new ARKit features with a newspaper box demo.

This week, the Times rolled the feature out as part of its Winter Olympics coverage. In Four of the World’s Best Olympians, as You’ve Never Seen Them Before, the publication spotlights figure skater Nathan Chen, speed skater J.R. Celski, hockey player Alex Rigsby, and snowboarder Anna Gasser. The results are impressive. I placed each athlete in my living room, then walked around them. From each angle, snippets of text about what I was seeing were overlaid on the image providing additional details and context. The app also makes use of haptic feedback on the iPhone to alert users to new information as they examine a scene.

Read more

Field Trips Just Got Cooler: Waypoint EDU

With the ubiquity of the smartphone in increasingly younger classrooms, integrating them into education is fast becoming a requirement for teachers. What better way to tame smartphones in the classroom than to make them part of the learning process?

Matthew Braun, developer of SketchParty TV (one of my favorite Apple TV games), released a new app to do just that. Waypoint EDU uses AR to make the phone not only a learning tool, but a truly interactive experience that can take place outdoors. Or anywhere.

From the student perspective, it works like this: students see a waypoint on a map of their current location and move around to find it. A la Pokemon Go, they search by looking through their phones, scanning for an out-of-place object such as a (miniature) colosseum sitting in a park. Once they’ve found the waypoint, they answer a quiz question to reveal the next waypoint. Think augmented reality geocaching.

From the teacher (or parent) side, creating a curriculum is pretty easy. I didn’t get into creating a fully customized one while I was trying it out, but editing the waypoints and related questions is simple. Once you have your curriculum set up, you just pull up the map and draw the playfield with your finger. The waypoints are automatically placed within the playfield, ready for the Hunt to be shared with the students via AirDrop. You can currently add artwork from a library, and the ability to add your own artwork will be a paid feature in a future update.

Waypoint EDU is a free app. Obviously, it has the requirement that everyone in the group has access to an iPhone. In situations where that’s possible, Waypoint EDU seems to me like the future of field trips. Below is a video of Waypoint EDU in action. You can find it on the App Store, and get more info at waypointedu.com.