This week's sponsor


The Code Editor for Your iPad and iPhone

Posts tagged with "camera"

Spectre: A Computational Approach to Long-Exposure iPhone Photography

Spectre is a new specialized camera app from the team that created Halide, one of our favorite camera apps on iOS. The Halide team describes Spectre as a computational shutter for the iPhone, which allows the app to do things like remove people from a crowded scene, create artistic images of rushing water, and produce light trails at night. The same sort of images can be created using traditional cameras, but getting the exposure right, holding the camera absolutely still, and accounting for other factors make them difficult to get right. With Spectre, artificial intelligence is used to simplify the process and make long-exposure photography accessible to anyone with an iPhone.

Read more

Apple’s Clips Introduces New Selfie Scenes, Filters, and More

On launch day for the new iPad Pros, which feature the iPhone’s TrueDepth camera system for the first time, Apple has upgraded its Clips video app with new features that take advantage of TrueDepth’s power. Today’s update also brings new camera filters, posters, stickers, and soundtrack options.

If you have an iPhone X, XS, or XR, or one of the new iPad Pros, the highlight of this release is a batch of six new Selfie Scenes. Last year when the iPhone X launched, Clips debuted Selfie Scenes as a fun and impressive way to utilize the new device’s TrueDepth camera system. Selfie Scenes isolate you from your environment and replace your surroundings with interesting digital backgrounds, such as the Millennium Falcon from Star Wars, or an animated world. The new scenes added in today’s update are called Clouds, High Noon, Big Backyard, Monster Lab, Animal Forest, and Municiberg Mayhem, a scene from The Incredibles 2. They’re all a lot of fun, providing different moods through sound effects and music. Apple says that Selfie Scenes perform better than ever on recent devices thanks to the A12 Bionic’s Neural Engine, and in my tests I did notice that the scenes were smoother and more responsive than before.

Filters and soundtracks are Clips’ next most substantial upgrades. There are three new filters: Comic Mono, Watercolor Mono, and Aged Film. Of these, the latter is easily my favorite, as the first two are only monochrome versions of existing filters. On the soundtrack side, there are a whopping 17 new tunes to choose from for your videos.

Every major Clips update adds a host of new posters, stickers, and text labels, and today’s is no exception. Continuing Apple’s partnership with Disney, there are poster options from Coco and The Incredibles 2, as well as designs related to sports, science, and more.

Though the app’s development cycle has slowed, Apple continues to plug away making Clips a great tool for short video creation. My biggest wish for the app – non-square video – has still gone unfulfilled, but hopefully one day we’ll get that change. Until then, the Selfie Scenes are a fun demo of the newest iPhones and iPad Pros, and Clips continues to be the most enjoyable video creation tool I’ve ever used. I think Apple’s on to something here, but the square video restriction continues to hold Clips back.

Halide Developer Ben Sandofsky Breaks Down How the iPhone XR Captures Depth Data

Ben Sandofsky from the team that makes the Halide iOS camera app has a detailed post on the iPhone XR’s camera and how Apple creates Portrait Mode photos with a single lens. Sandofsky walks through how Apple uses Focus Pixels to develop a rough Disparity Map and combines that with a Portrait Effects Matte to create Portrait Mode images.

The results have some advantages, but also distinct disadvantages compared to the iPhone XS’s camera. As Sandofsky explains:

It seems the iPhone XR has two advantages over the iPhone XS: it can capture wider angle depth photos, and because the wide-angle lens collects more light, the photos will come out better in low light and have less noise.


…most of the time, the XS will probably produce a better result. The higher fidelity depth map, combined with a focal length that’s better suited for portraiture means people will just look better, even if the image is sometimes a bit darker. And it can apply Portrait effects on just about anything, not just people.

Although Apple’s Camera app can only take Portrait Mode photos of people on the iPhone XR, the upcoming Halide 1.11 update will combine the XR’s Disparity Map and Halide’s own blur effect to apply a similar effect beyond human subjects. Sandofsky admits that the feature isn’t perfect due to the low quality of the Disparity Map created by the XR, but the photos included in his post show that it can take excellent pictures under some conditions.

It’s remarkable what is being done to squeeze depth information out of the XR’s single lens and instructive to understand how the underlying technology works. It’s also apparent that Apple has made significant advancements since the introduction of the first dual-lens cameras.


Halide Introduces Smart RAW for iPhone XS, Joining a Host of Other Improvements

Since its debut Halide has been one of the best manual camera apps available on iPhone. The month of September brought a number of challenges to Halide’s team though, thanks to all the photography work Apple put into iOS 12 and the iPhone XS. And in response, within the span of a few weeks Halide has receive two major updates: version 1.9 on iOS 12’s release date, and releasing today is version 1.10 featuring Smart RAW.

Read more

An In-Depth Explanation of Computational Photography on the iPhone XS

Outside of Apple employees, one of the people most knowledgeable about the iPhone’s camera is Sebastiaan de With, designer of the manual camera app Halide. It is fitting, then, that Sebastiaan would publish what I believe is the best explanation of the iPhone XS camera system to date. Following up on a piece he wrote about the new camera’s hardware changes, the subject of today’s article is software – specifically, all the work of computational photography on the iPhone XS and XS Max.

The piece starts with an explanation of the iPhone’s new Smart HDR feature, then details the exact reasons why selfies on the new iPhones appear to employ skin smoothing (a theory he soundly debunks). Finally, Sebastiaan details the problem that the XS camera poses for RAW camera apps like Halide, and shares about the forthcoming solution Halide’s team came up with: something they call Smart RAW.

There are too many excellent, informative tidbits to quote here, so I highly recommend you check out the article in full. This year’s iPhones are so full of interesting changes to the way the camera works, most of which are undocumented by Apple – as Sebastiaan says, it is “a whole new camera” in many ways.


iPhone XS Line Capturing Selfies with the Appearance of Skin Smoothing

The iPhone XS and XS Max have been in the hands of users for a week now, and during that time many selfies have undoubtedly been taken on the new devices. Some users have noticed an unexpected difference with their selfies, however: their skin looks smoother and less realistic than it should. Juli Clover reports on this for MacRumors:

When taking a selfie in a situation where lighting is less than ideal, such as indoors or outdoors in areas with lower lighting, the iPhone XS Max appears to be applying a drastic smoothing effect that can hide freckles, blemishes, and other issues.

In full outdoor lighting the problem is less apparent, which has led to speculation that the skin smoothing is actually a result of some heavy-handed noise reduction techniques.

You can test the new camera yourself with an iPhone XS Max and an older iPhone like an iPhone X model by taking selfies indoors and outdoors and comparing the differences between the two. In almost all cases where the lighting is low or uneven, photos captured with an iPhone XS Max look dramatically different.

9to5Mac has a quote from one user, Abdul Dremali, who said, “Apple reached out to me yesterday and are working on this issue actively.” That isn’t necessarily confirmation that a skin smoothing effect is being applied, but it does indicate that something’s happening with the XS front-facing camera that wasn’t intended.

There’s often one controversy or another surrounding the launch of a new iPhone, such as antennas not working properly or iPhones bending under pressure. This issue doesn’t seem quite like those past problems, as it should be a quick fix in software if something’s not functioning correctly. In 2018 there’s a lot that happens in software every time a photo is taken on an iPhone; it wouldn’t surprise me if a small tweak arrives in iOS 12.0.1 that helps skin remain natural to the eye no matter the lighting conditions. One way or another, we should hear more from Apple before long.


Austin Mann’s Review of the iPhone XS Cameras

In addition to the early slate of iPhone reviews from the press, it’s become tradition in recent years for each iPhone to be graded as a camera by professional photographer Austin Mann. I especially enjoyed Mann’s review this year of the iPhone XS camera system. He writes:

Most of the time my expectations for camera upgrades on “S” years aren’t so high, but after shooting with the iPhone XS for a week, I can confidently say it’s a huge camera upgrade. There’s a lot of little improvements, but Smart HDR definitely takes the cake. This is a feature and technology that improves virtually everything you capture with your iPhone camera. I think you’ll be really thrilled when you experience the results yourself.

As I shared in last week’s issue of MacStories Weekly for Club MacStories, the iPhone XS and XR announcements caught me by surprise in that I expected there to be more change in the devices compared to last year’s iPhone X. I’ve ordered a XS Max, but the primary reason for my upgrade was the additional screen real estate compared to my X; bigger display aside, September’s keynote didn’t provide much of a compelling reason for me to purchase a new phone this year. However, Mann’s review and that of John Gruber have helped provide much-needed additional detail on the camera upgrades in the XS, which sound impressively significant.

One of the standout lines in Mann’s review for me comes near the beginning, where he says, “iPhone XS captures what your eyes see.” It’s hard to find higher praise than that.


The iPhone XS’ Camera and Neural Engine

Apple describes the XS as sporting “dual 12MP wide-angle and telephoto cameras”. This will be obvious to most of you, but in case it’s not, they’re not just dual rear-facing lenses, they’re dual rear-facing cameras. The wide-angle and telephoto lenses each have their own sensors. As a user you don’t have to know this, and should never notice it. The iPhone XS telephoto camera is the same as in the iPhone X — same lens, same sensor.

But the iPhone XS wide-angle camera has a new lens, which I believe to be superior to last year’s, and an amazing new sensor which is remarkably better than last year’s. And last year’s was very good.

Anytime an iPhone review gets too technical about camera details and photography lexicon, I tend to gloss over it and move on. I’m not a camera expert and I usually don’t care about the nitty-gritty. But John Gruber’s analysis of the iPhone XS’ camera stack, A12 SoC, and seemingly unadvertised improved sensor is one of the most interesting camera-focused iPhone reviews I’ve read in years. I don’t want to spoil it – move past the photos at the beginning and keep reading.