Media Outlets Use ARKit to Bring the News into Your Living Room

Last week, The New York Times announced that it had added an augmented reality feature to its iOS app. The first article with embedded AR content was a preview of the feature published last week that explained to readers how it worked. At the bottom of the article was a newspaper box that could be dropped into your surroundings. I showed it off to some friends over the weekend, and everyone was impressed by how realistic it looked as they walked around the box in a neighbor’s kitchen.

The New York Times debuted its new ARKit features with a newspaper box demo.

This week, the Times rolled the feature out as part of its Winter Olympics coverage. In Four of the World’s Best Olympians, as You’ve Never Seen Them Before, the publication spotlights figure skater Nathan Chen, speed skater J.R. Celski, hockey player Alex Rigsby, and snowboarder Anna Gasser. The results are impressive. I placed each athlete in my living room, then walked around them. From each angle, snippets of text about what I was seeing were overlaid on the image providing additional details and context. The app also makes use of haptic feedback on the iPhone to alert users to new information as they examine a scene.

The feature currently works only on compatible iOS devices running iOS 11, though an Android version is in the works. A rough approximation of the AR experience is available on the web too, though I found that the scrolling stuttered too much, making it hard to navigate the article. Although the first implementations of ARKit in the Times’ iOS app feel more like proofs of concept than journalism, it’s an interesting start that has the potential to offer a new perspective on events.

Unsurprisingly, a big component of the Times’ interest in AR is also advertising. According to a story in Ad Week:

[Graham] Roberts[, the Times’ head of immersive platform storytelling,] is excited about the advertising possibilities of AR. “You can see what the car is like in your driveway. You can see what the coffee machine is like on your counter before you bought it. Walk around it,” he said. “But I think the foundation there is the fact that it can come within the article experience. So it’s frictionless, really effectual and it gives you a thing that you can do that was not possible before. And I think that that will put the New York Times in a really unique position for advertising.”

The Rosetta Stone, the Berlin Wall, and the Falcon Heavy rocket with scale information in Quartz.

The New York Times isn’t alone. In fact, Quartz added ARKit to its iOS app back in September 2017 as soon as iOS 11 became available. Among the objects available for viewing in Quartz’s app are SpaceX’s Falcon Heavy rocket, the Rosetta Stone, and the Berlin Wall. Each item can be resized and rotated with your fingers on the screen of an iOS device, something The New York Times’ app cannot do. Quartz also provides scale information so you can get a sense of how big each object is.

It will be interesting to see to what degree AR takes off with news outlets. It can’t be cheap to produce AR experiences – the Times’ Winter Olympics piece lists 17 people who were involved in its production in one capacity or other. Still, AR has the potential to bring a new perspective on the news that wasn’t possible before and offer new ways for advertisers to reach consumers. For now, however, ARKit features are primarily a novelty that will draw attention to stories but need to be integrated into publications’ reporting better before they are likely to become commonplace.