Live Text and Data Detectors

I tweeted about this a few weeks ago, and, as you can see, people seem to be really into these changes coming to iOS 16.

As part of Apple’s efforts to regularly improve the “intelligent” features of our devices, iOS 16 is gaining built-in, system-wide conversion for all kinds of units in every app that has a native text field or supports data detectors. In practice, this means that iOS 16 can recognize when you’re typing a currency value, a time zone, temperature, or even the weight and length of something, underline it, and offer you to instantly convert it to another popular unit.18

For instance, if you type “35 C” in Messages, the other person on an iPhone running iOS 16 will be able to tap the value and get an instant conversion to Fahrenheit. Same if you receive a message that contains a currency, weight of something in grams, and so forth. The popup includes the ability to copy converted values, and it even supports disambiguation for multiple values if it’s not sure of which unit you want to start with. If you just type “21 degrees”, for instance, iOS 16 will display an additional menu to pick between Celsius, Fahrenheit, or actual degrees. In another amazing detail, the icons used for different temperature values actually have meaning and are based on the perceived temperature.

iOS 16's unit conversion feature in Messages, powered by data detectors.

iOS 16’s unit conversion feature in Messages, powered by data detectors.

This feature also works for time zones: if someone sends you an email and they mention “5 PM PT” but you live in Europe, you’ll be able to tap the time zone and see what that time corresponds to in your time zone. As someone who’s been working for over 13 years with people based in the United States and who’s often struggled with time zone converter apps, I can’t emphasize this enough: built-in time zone conversion for text (Siri could already do this) is going to save me a lot of time.

Even better, all these built-in conversions aren’t limited to apps like Notes and Messages where you can tap on detected units to convert them. If you’re in an app like Twitter or Safari and manually select text, the redesigned edit menu of iOS 16 will show a snippet with converted values as well. To do this, just select the text with the value you want to convert, swipe left inside the edit menu, and you’ll get your conversion there too.

Converted units in the edit menu for selected text.

Converted units in the edit menu for selected text.

iOS 16’s support for unit conversion reaches deep into the operating system at multiple levels: it’s supported in data detectors, the edit menu, Siri, in Visual Look Up, and even in Translate’s new Camera mode. It’s probably too early to tell whether Apple has Sherlocked multiple categories of converter apps with these enhancements, but one thing’s for sure: since I discovered this, I haven’t used any third-party app or shortcut for unit conversions.

Speaking of Live Text, in addition to supporting the aforementioned data detectors, Live Text is also supported in video frames now. Watch a video, pause it, and if there’s text in the frame, you’ll be able to interact with it using Live Text. This entire feature continues to feel incredible, and with support for data detectors, translation, and videos in iOS 16, I’m amazed by the quality work that the Live Text team at Apple is putting out on an annual basis.

And More…

Add credit card CVV to autofill info. For years now, Safari has been able to automatically fill credit card information on webpages by using the cardholder’s name, card type, number, and expiration date previously configured in Settings. The only field that couldn’t be auto-filled was the card’s security code (or CVV), which you had to manually type each time you were using a credit or debit card in Safari.

Adding the CVV to a credit card in Autofill settings.

Adding the CVV to a credit card in Autofill settings.

This is changing with iOS 16, which allows you to (optionally) store the CVV code in Autofill’s settings. To do this, you’ll have to open Settings ⇾ Safari ⇾ Autofill, select an existing credit card, then tap ‘Edit’ in the top-right corner of the screen to enter editing mode. This way, you can add the card’s verification code and you won’t be asked to manually type it again the next time you’re purchasing something online.

Keyboard haptics. iOS 16 introduces an Android-like keyboard haptic mode that makes each key press on the iPhone’s software keyboard result in subtle haptic feedback from the Taptic Engine. You can enable this in Settings ⇾ Sounds & Haptics ⇾ Keyboard Feedback. I turned on this option months ago after the first developer beta of iOS 16, and I haven’t looked back since. Typing without haptic feedback would feel broken on my iPhone now; it brings a new layer of tactility to the keyboard that makes it more enjoyable to type on a small screen.

Wallpaper crop when cropping photos. When cropping an image in the Photos app, you’ll find a new format to crop images to a wallpaper size that fits your devices. That’s another way Apple is making it easy for you to create Home and Lock Screen wallpapers this year.

Clipboard permissions when pasting. In iOS 16, Apple is cracking down on apps that automatically read the contents of your clipboard when you open them by displaying an alert whenever an app tries to access the clipboard without your consent, from which you can allow or deny access:

Prepare to see this alert in a lot of your favorite apps.

Prepare to see this alert in a lot of your favorite apps.

Now, I’m all in favor of Apple making tools to protect our privacy and ensuring that we always have control of our data, but in doing this, I should also note that Apple has made apps that had legitimate reasons to read the system clipboard more annoying to use – and less useful – in iOS 16. An app like Parcel, for example, could automatically read your clipboard and suggest adding a new delivery for an item if it found a valid tracking number in your clipboard. That process is not so seamless anymore in iOS 16, and the same is true for other apps that would offer to do something upon reading the clipboard, such as Fantastical (for dates) and Matter (for articles).

Apple recommends developers use a native ‘Paste’ button in lieu of automatic clipboard monitoring, but I disagree with their approach.

Opener is an example of an app that has switched to Apple's recommended Paste button.

Opener is an example of an app that has switched to Apple’s recommended Paste button.

There should be an API to grant a specific app automatic, persistent clipboard access by authorizing it once. Instead, whether I’m using Matter or Obsidian (where all ‘paste’ commands trigger this alert over and over now), I feel like I’m constantly being punished by doing something I know, and I’m just annoyed by the alert at this point. I wish I could turn it off. Hopefully, with enough feedback, Apple will come up with a better solution for developers and users next year. The solution isn’t to punish users with incessant alerts: it’s to give developers new and modern clipboard APIs.

Show your Wi-Fi password. File this under “I have no idea how we went so long without it”: in iOS 16, you can see the password of the Wi-Fi network you’re currently connected to. Open your Wi-Fi settings, tap on the Password field, then authenticate with your device’s preferred method, and the password will be revealed to you along with a Copy button. This has already come in handy for all those times when I needed to re-enter my Wi-Fi password on non-Apple devices such as my gaming PC, Nintendo Switch, or Steam Deck. Seriously, how is this only shipping in iOS 16?

Landscape Face ID. Although we still don’t have a proper landscape Lock Screen for iPhone, this iOS 16 tweak is going to prove valuable for all those times when you want to unlock your device while laying in bed (or looking at it sideways, I guess). Face ID now works in landscape mode. The feature seems to be limited to the iPhone 13 line, and it works as advertised.

New settings for measurement system and first day of the week. European users who like to keep your iPhones in American English, rejoice: in Settings ⇾ General ⇾ Language & Region, you can now override values for the measurement system and first day of the week to always be, say, Metric and Monday, respectively. This is exactly what I did to keep my iPhone set to English, use the U.S. App Store, but also live with a default measurement system that, you know, makes sense.

Dedicated AirPods settings page. The long national nightmare of hidden AirPods settings on iPhone is finally over with iOS 16: when AirPods are connected to your device, a top-level page appears in the Settings app, allowing you to quickly jump into your AirPods preferences.

No more need to go find connected AirPods in Bluetooth settings.

No more need to go find connected AirPods in Bluetooth settings.

This way, you’ll no longer have to go find your connected AirPods under Bluetooth settings. In another nice touch, the AirPods page comes with a preview of the model you have at the top.

iCloud backup over LTE and 5G. You know how much Apple loves 5G. We all love 5G. Tim Cook especially. If you have 5G you’d be happy to hear that iOS 16 now lets you perform iCloud backups over 5G. And LTE connections, too, I guess. But nobody cares about those schmucks without 5G anymore, right?

New Memoji. As always in a new version of iOS, there are some improvements to Memoji. This year’s addition include six new poses for stickers, 17 new and updated hairstyles, more nose types and neutral lip colors, and more. As an Italian, I am almost contractually obligated to mention that the best Memoji pose has been delivered to us in iOS 16, and you know what it is:

  1. Quick actions for data detectors are available on devices with the A12 Bionic SoC and later. They're supported in English, Chinese, French, Italian, German, Japanese, Korean, Portuguese, Spanish, and Ukrainian text. ↩︎

Supported by MacStadium: Cloud solutions to simplify Mac for business.

Add Bookmark

Use this link to save your spot in the review and return to it later.
You can add this as a bookmark or share it with friends.