THIS WEEK'S SPONSOR:

Daylite

Do Big Things With Your Small Business


Posts tagged with "AR"

Kara Swisher Interviews Apple CEO Cook for Sway

Apple CEO Tim Cook was interviewed on Sway, The New York Times’ podcast hosted by Kara Swisher, in an episode released today. Swisher asked Cook about a wide range of topics, including privacy, iOS 14.5, Parler’s removal from the App Store, autonomous vehicles, AR, its upcoming court case with Epic Games, and more.

On privacy and the reaction to App Tracking Transparency, Cook said he was shocked by the degree of pushback the feature has caused. Asked whether he thought ATT will harm businesses that rely on digital advertising, Cook said:

I think that you can do digital advertising and make money from digital advertising without tracking people when they don’t know they’re being tracked. And I think time will prove that out. I’ve heard this about other things we’ve done in the past that it’s almost existential and it wasn’t. I don’t buy that.

Regarding Parler’s removal from the App Store, Cook explained that can return to the App Store when they comply with its rules:

Well, in some ways, it was a straightforward decision, because they were not adhering to the guidelines of the App Store. You can’t be inciting violence or allow people to incite violence. You can’t allow hate speech and so forth. And they had moved from moderating to not being able to moderate. But we gave them a chance to cure that. And they were unable to do that or didn’t do that. And so we had to pull them off. Now, having said that, Kara, I hope that they come back on. Because we work hard to get people on the store, not to keep people off the store. And so, I’m hoping that they put in the moderation that’s required to be on the store and come back, because I think having more social networks out there is better than having less.

Cook also made the case that human curation on the App Store is a crucial element of the marketplace’s security model, rejecting the notion that users should be able to sideload apps and elaborating:

I think curation is important as a part of the App Store. In any given week, 100,000 applications come into the app review. 40,000 of them are rejected. Most of them are rejected because they don’t work or don’t work like they say that they work. You can imagine if curation went away, what would occur to the App Store in a very short amount of time.

Regarding new products, Cook wouldn’t confirm whether Apple is planning to offer augmented reality hardware or an autonomous car. Still, his excitement about those underlying technologies was evident, noting that AR, in particular, is critical to Apple’s future.

Also of note was Cook’s comment that iOS 14.5 is ‘just a few weeks’ away, which is longer than I expected and perhaps a sign that an April product event will occur.

The interview, which is just under 36 minutes long, touches on other topics, including Apple’s role in policy issues like voting rights, working with the US government, and Cook’s role as the first openly gay CEO of a Fortune 500 company. The episode is available in Apple Podcasts as well as third-party podcast players, and The New York Times has published a transcript of the entire interview.

Permalink

Halide Team Experiments with iPad Pro’s LiDAR Scanner

Source: Halide Blog

Source: Halide Blog

Sebastiaan de With, on the Halide blog, goes deep on the 2020 iPad Pro’s camera module. His examination reveals that the device’s wide camera is virtually identical to that of the 2018 model. And the ultra-wide camera, unfortunately, isn’t quite up to the quality level of what’s found in the iPhone 11 and 11 Pro.

The most exciting and impressive aspect of the camera system is the LiDAR Scanner. The Halide team actually went to the trouble of building an entire proof of concept app that utilizes the LiDAR Scanner to capture your surroundings.

With Halide, we’d love to use the depth data in interesting ways, even if it’s low resolution. There was only one problem: there are no APIs for us as developers to use to get access to the underlying depth data. They only expose the processed 3D surface.

What if we re-thought photographic capture, though? We built a proof-of-concept we’re calling Esper.

Esper experiments with realtime 3D capture using the cameras and LIDAR sensor at room scale. It’s a fun and useful way to capture a space.

I always love reading de With’s in-depth explanations and comparisons of new iPhone or iPad cameras, and this was an especially fun one.

Permalink

Harry Potter: Wizards Unite AR Game Launches on iOS

It was the summer of 2016 when Pokémon GO took the world by storm, and now, almost exactly three years to the day later, there’s a new AR game trying to capture the same magic. Harry Potter: Wizards Unite launched today in the US, UK, and select other countries from the same studio behind Pokémon GO, Niantic.

At the core of Wizards Unite is the same AR-based system employed in Pokémon GO, whereby you can explore a digital world that’s mapped to the real world around you, where different real-life locations are mapped to in-game hot spots for engaging with certain game elements. Special locations in the game, such as fortresses, are designed for teams of players to conquer together; combined with the AR world exploration mechanic, this encourages strong community engagement in the game. If Wizards Unite takes off in any way like Pokémon GO, expect to see bands of players roaming the streets with their phones at the ready.

Chronologically, Wizards Unite takes place following the original seven book Harry Potter series but still includes many of those original characters. In the game’s tutorial, for example, a slightly aged Harry Potter, now working as an auror, instructs you regarding a crisis the Ministry of Magic is currently facing that needs your assistance. This sort of tight integration with the world and characters Harry Potter fans know and love, combined with the community-centric element found in games like Pokémon GO, could make Wizards Unite a really special experience for anyone who loves J.K. Rowling’s wizarding world. As one of those people myself, even though I never got into Pokémon GO, I’m excited to spend some quality time trying out Wizards Unite.

You can download Harry Potter: Wizards Unite on the App Store now.


Nike Introducing AR Shoe Sizing Feature This Summer

Edgar Alvarez reports for Engadget on an upcoming AR feature I can’t wait to try:

Nike has been experimenting heavily with augmented reality for a few years now, and the company is continuing to work on new experiences powered by the technology. The sportswear giant is now introducing Nike Fit, a feature that uses a combination of computer vision, scientific data, artificial intelligence and recommendation algorithms to scan your feet and find the right shoe fit for you. And you can do it all in augmented reality, using the Nike app on your smartphone. Nike says that, according to industry research, over 60 percent of people wear the wrong size shoes. With Nike Fit, the company is hoping to solve that problem.

The AR experience itself is fairly simple: You open up the Nike app, go to a product page and, next to where there’s usually a menu that lets you pick the size of your shoes, you’ll see a new option to measure your feet. From there, the camera will pop up and you’ll be asked to stand next to a wall and point your smartphone at your feet, which will prompt a view that uses two AR circles to level your phone. Once the feature recognizes your feet and your physical environment, it starts scanning your feet and then tells you your ideal shoe size for Nike footwear. The entire process takes less than a minute.

It’s been two years since Apple introduced ARKit, yet there are still very few AR apps I’ve found add meaningful value to my life. Outside of measuring apps, the only ARKit experiences I’ve enjoyed related to shopping for furniture and other home accessories. It looks like shoe shopping will be the next major area where AR becomes more than a fun demo. According to Alvarez, the Nike Fit feature will be added to the Nike app this July.

Permalink

Niantic Previews Harry Potter: Wizards Unite, the Studio’s Follow-Up to Pokémon GO

Later this year the team at Niantic will launch a new game based on a very popular IP: Harry Potter. The game’s full title is Harry Potter: Wizards Unite, and it will come to both iOS and Android as an AR-centered gaming experience in the vein of Pokémon GO, Niantic’s biggest hit.

Today Nick Statt of The Verge published his impressions of Wizards Unite following a preview Niantic hosted recently. Statt writes:

The core activity in Wizards Unite involves collecting a mix of artifacts and sometimes saving notable characters from the series who are stuck in a bad situation, like the titular Harry being pinned down by a spooky dementor. From there, you play a little tracing mini-game, to evoke the casting of a spell and then collect the item or free your friend. Afterward, you’re able to add that item or character to your collections book, earn experience points, and level up.
[…]
Beyond traversing the map, collecting those artifacts, and visiting inns to eat food, players of Wizards Unite will have a few more advanced activities to keep them busy. Those include leveling up your character, picking a subclass (called a “profession”) to learn new abilities, and then teaming up with up to five other players to compete in a fortress (what Niantic has designed to be this game’s version of gym battles from Pokémon Go). These team challenges feel like a cross between a traditional strategy game and something similar to Nintendo’s Fire Emblem, where one-on-one battles take place simultaneously after players select an enemy from a top-down map.

Based on Statt’s article, Harry Potter: Wizards Unite is going to have a lot more depth and complexity to it than Pokémon GO, which I’m excited about. Pokémon GO has scaled over time to become more of a gamer’s game than it used to be, but with Wizards Unite it sounds like that added depth will be there from the start for those who want it.

There’s still no firm release date available for Wizards Unite, but a 2019 launch is confirmed.

Permalink

Media Outlets Use ARKit to Bring the News into Your Living Room

Last week, The New York Times announced that it had added an augmented reality feature to its iOS app. The first article with embedded AR content was a preview of the feature published last week that explained to readers how it worked. At the bottom of the article was a newspaper box that could be dropped into your surroundings. I showed it off to some friends over the weekend, and everyone was impressed by how realistic it looked as they walked around the box in a neighbor’s kitchen.

The New York Times debuted its new ARKit features with a newspaper box demo.

The New York Times debuted its new ARKit features with a newspaper box demo.

This week, the Times rolled the feature out as part of its Winter Olympics coverage. In Four of the World’s Best Olympians, as You’ve Never Seen Them Before, the publication spotlights figure skater Nathan Chen, speed skater J.R. Celski, hockey player Alex Rigsby, and snowboarder Anna Gasser. The results are impressive. I placed each athlete in my living room, then walked around them. From each angle, snippets of text about what I was seeing were overlaid on the image providing additional details and context. The app also makes use of haptic feedback on the iPhone to alert users to new information as they examine a scene.

Read more


Field Trips Just Got Cooler: Waypoint EDU

With the ubiquity of the smartphone in increasingly younger classrooms, integrating them into education is fast becoming a requirement for teachers. What better way to tame smartphones in the classroom than to make them part of the learning process?

Matthew Braun, developer of SketchParty TV (one of my favorite Apple TV games), released a new app to do just that. Waypoint EDU uses AR to make the phone not only a learning tool, but a truly interactive experience that can take place outdoors. Or anywhere.

From the student perspective, it works like this: students see a waypoint on a map of their current location and move around to find it. A la Pokemon Go, they search by looking through their phones, scanning for an out-of-place object such as a (miniature) colosseum sitting in a park. Once they’ve found the waypoint, they answer a quiz question to reveal the next waypoint. Think augmented reality geocaching.

From the teacher (or parent) side, creating a curriculum is pretty easy. I didn’t get into creating a fully customized one while I was trying it out, but editing the waypoints and related questions is simple. Once you have your curriculum set up, you just pull up the map and draw the playfield with your finger. The waypoints are automatically placed within the playfield, ready for the Hunt to be shared with the students via AirDrop. You can currently add artwork from a library, and the ability to add your own artwork will be a paid feature in a future update.

Waypoint EDU is a free app. Obviously, it has the requirement that everyone in the group has access to an iPhone. In situations where that’s possible, Waypoint EDU seems to me like the future of field trips. Below is a video of Waypoint EDU in action. You can find it on the App Store, and get more info at waypointedu.com.


Amazon Adopts ARKit in iOS Shopping App for Select Products

In an update released for its iOS shopping app, Amazon has introduced a new way of viewing items from the online retailer: AR View. Built on Apple’s ARKit technology in iOS 11, AR View provides shoppers with a better understanding of how products will look when placed inside their homes.

AR View is accessed inside the Amazon app by tapping the camera button, then selecting AR View from the assortment of camera options. You’ll then get to browse through a limited selection of product categories, such as Living Room, Kitchen, and Electronics; there’s also a Top Picks section. Unlike the similar AR experience from IKEA Place, only one product can be previewed in Amazon’s AR View at a time. After placing a product in AR, you can move its position or rotate it, and pressing the button with three dots will take you to the full product page for initiating a purchase.

Amazon claims that thousands of items are available in AR View, but currently only a fraction of that estimate appears for me inside the app; we should except the number of AR-compatible items to grow over the coming holiday shopping season. It also wouldn’t be surprising to see AR View roll out to other parts of the app in the future, such that if you’re viewing the product page for an Amazon Echo, for example, there will be a button that allows you to instantly view the item in AR.

Today’s version of AR View is a first step toward enhancing the Amazon shopping experience with AR. There’s plenty more work to be done, but it’s exciting to see a glimpse into how transformative AR can potentially be for online shopping.


Made with ARKit

ARKit is one of the iOS 11 features I’m really excited about along with iPad improvements, changes to Notes, and better screenshot workflows. The “problem” with ARKit is that Apple isn’t offering a proper AR app for iOS yet – it’s a framework for developers to create AR experiences. Thus, until you play with an ARKit demo, it’s hard to understand the extent of Apple’s efforts and the potential for future ARKit-enabled apps.

Fortunately, the folks at Made with ARKit have been collecting early demoes shared by developers showing a variety of AR apps that will be possible later this year. And some of these are already incredible. From a rocket landing in your backyard to robots dancing in your living room and obvious measuring tapes, these videos give us an early glimpse at the promise of ARKit and the quality of tracking and rendering on an iPhone’s screen.

I don’t know if these showcases will turn into actual shipping products this Fall, but I have a feeling this new category of apps will become a great reason for millions of users to upgrade to iOS 11 quickly. I can’t wait to play with some of these AR apps.

Permalink