Digital Touch and Handwriting

When Apple introduced Digital Touch on watchOS in 2014, it was safe to assume it’d eventually find its way to iOS. Two years later, Digital Touch has been built into Messages in iOS 10, gaining a prominent spot between photos and the new iMessage App Store.

Digital Touch can be activated from the heart icon with two fingers – a reminder of its Apple Watch legacy. Tapping the button turns the lower half of the screen into an interactive pad where you can draw, send taps and heartbeats, and annotate photos and videos.

Digital Touch has three sections: a color picker along the left side (where, like on the Watch, you can long-tap a color to pick another one); a drawing area in the middle; and icons explaining Digital Touch features rotating on the right. At the bottom, a chevron lets you open Digital Touch in expanded mode, taking over the conversation in full-screen.

There isn’t much to say about the functionalities adapted from watchOS. Sketches are easier to create thanks to the bigger screen, though I think that, to an extent, the constraints of the Watch incentivized creativity. Also, sketches look like images with a black background pasted into conversations: they’re animated, but they don’t feel as integrated as they used to be on the Apple Watch. They look like simple image attachments on iOS 10.

Taps and heartbeats are the kind of features someone decided to bring over to iOS so they wouldn’t go to waste. They fundamentally feel out of place on iOS given the lack of haptic feedback on the wrist and their black background.

When you receive a tap from someone on the Apple Watch, you feel tapping on your wrist. Taps are animated images on iOS 10 and there’s nothing special about them. The physical connection is lost. Apple could have made taps part of the conversation view, letting them ripple through bubbles like effects do, or use vibration as feedback; instead, they settled on GIFs.

Heartbeats are even more baffling, as they aren’t “real” heartbeats due to the lack of a heart rate sensor on iOS. When you hold two fingers on the screen to send your heartbeat on iMessage17, iOS generates a generic animation that isn’t a representation of anyone’s heartbeat. The sense of intimacy watchOS fostered thanks to Digital Touch and its heart rate sensor – of knowing that the heartbeat animation represented the actual beating heart of a friend or partner – isn’t there on iOS.

And don’t get me started on the sadness of swiping down with two fingers to send a heartbreak.

Then there’s 3D Touch, which is used in Digital Touch to send “fireballs”. If you press on the Digital Touch pad, iOS 10 creates a pulsing fireball that will be sent as an animated image.

That's a fireball.

That’s a fireball.

I’m not sure what to make of the fireball – does sending one show you’re thinking of someone? That are you’re upset with them? That you’ve realized 3D Touch exists in iMessage? Is it a reference to John Gruber? It’s an open-ended question I’ll leave to the public to resolve.

The standout Digital Touch feature is one that has been built around the iPhone’s hardware. Tap the video icon, and you’ll bring up a camera UI to sketch on top of what the camera is seeing. You can also add Digital Touch effects in real-time while recording a 10-second video (to take a picture, tap the shutter icon).

Effects with the Digital Touch camera are fun.

Effects with the Digital Touch camera are fun.

The combination of sketches and kisses with videos is fun and highly reminiscent of Snapchat; I’ve been using it to send short clips with funny/witty comments or sketches drawn on top of them. Apple should add more iOS-only “stamps” or animations to Digital Touch for photos/video without copying what they’ve done on watchOS.18

Unrelated to Digital Touch, but still aimed at making conversations more personal, is handwriting mode.

Anyone who’s familiar with handwritten signatures in Preview and Markup will recognize it: handwriting can be accessed by tapping the ink button on the iPad keyboard or turning the iPhone sideways. It opens an empty area where you can handwrite a message in black ink using your finger (or Apple Pencil). There’s a list of default and recent messages at the bottom (which can be deleted by long-tapping them), and no additional controls.

How handwritten messages look in conversations.

How handwritten messages look in conversations.

I found handwriting mode to be nicer than Digital Touch. Handwritten messages aren’t contained in a black image and ink animates beautifully19 into the conversation view, which creates the illusion that someone has written a message for you inside Messages instead of sending an image attachment. It’s a better integration than Digital Touch.

Digital Touch on iOS 10 could have used more work. Features that had some reason to exist on watchOS’ hardware have been lazily ported to iOS, removing the physical interaction and feedback mechanism that made them unique on the Watch.

I’m not sure the iOS Digital Touch we have today is worth giving up a premium slot as a default iMessage app next to the Camera. It’s a “Friends button” scenario all over again. I wouldn’t be surprised if that permanent placement becomes customizable next year.

Tapback

iOS 10 brings new options to react to messages, too.

Called Tapback, the feature is, essentially, Apple’s take on Facebook’s redesigned Like button and Slack’s reactions. If you want to tell someone what you’re thinking without texting back, you can double tap20 a message – any kind of bubble – to bring up a menu with six reactions: love, thumbs up, thumbs down, ha-ha, exclamation points, and question mark.

Sending a Tapback.

Sending a Tapback.

The interaction of Tapback is delightful. Icons animate when you tap on them, and they play a sound effect once attached to a message. You can’t create your own reactions by picking any emoji like on Slack, but, looking at a conversation with a bunch of hearts, thumbs-ups, and ha-has, the feeling is the same.

TapbackReplay

Tapbacks are especially effective in group threads where everyone can “vote” or express their immediate reactions without typing. A Tapback can be changed at any point during a conversation, but you can only leave one reaction per message.

If what happened in my Slack teams over the past year is of any indication, Tapback should become a useful way to let someone know you’ve acknowledged or liked their message without writing anything back.

Emoji

Slack’s influence on iMessage has propagated to emoji as well. Messages that only contain emoji (no text) will be sent as big emoji (at 3x), so you can truly appreciate the details that make up Apple’s most popular characters.

Regular and big emoji.

Regular and big emoji.

I’ve been a fan of jumbo emoji since Slack rolled them out last year. They’re a perfect fit for iMessage. Emoji are expanded in the text field before sending them – I chuckle every time I see a big thinking face about to enter a conversation. Messages will only display up to three big emoji at a time; if you create a message containing four emoji, they’ll be sent at normal size.

Emoji improvements don’t stop there. Apple must have noticed that users like to write messages and replace words inside them with appropriate emoji, and they’re introducing an option to automate the process in iOS 10. Possibly, this innocuous feature (which only works in Messages) is even going to power Apple’s Differential Privacy for crowdsourced data collection.

If you write a message in iOS 10 and then open the emoji keyboard, the system will scan words you’ve entered in the text field and try to match them up with emoji. If a related emoji is found, a word will be highlighted in orange. Tap it, and it’ll be replaced with the emoji.

Tap the emoji keyboard to replace words with emoji.

Tap the emoji keyboard to replace words with emoji.

If multiple emoji options are available for a single word, tapping it opens a menu to choose one.

Multiple emoji options.

Multiple emoji options.

I’m not exactly the target audience for this feature (I either only send emoji or put some next to a word), but I recognize that a lot of people treat emoji as substitutes for words. Apple devised a clever and thoughtful way to “emojify” text, letting the OS compensate for a search box still missing from the emoji keyboard.

Under the hood, emoji replacements hinge on a system that has to build up associations and trigger words, follow trends, and adapt for international users and different meanings of the same emoji around the world. Based on what Apple has revealed about Differential Privacy, data on emoji picked by users will be collected in aggregate to improve the accuracy of suggestions.

My understanding is that Apple started from a set of words curated from common expressions and Unicode annotations, and began scaling to millions of users and dozens of languages for over 1800 emoji during the iOS 10 beta stage. In my case, emoji replacements worked well for both English and Italian.

Crowdsourcing this aspect of iMessage makes sense given the popularity and many meanings of emoji. It’ll be interesting to see how suggestions will be refined as iOS 10 usage picks up.


  1. You can also tap with two fingers to send kisses, which weren't available on watchOS before. ↩︎
  2. Perhaps even open up selfie effects to developers? ↩︎
  3. A detail that I love: look closely at how ink spreads out on the "page" once it's absorbed. Realistic. ↩︎
  4. You can also tap & hold a message to show Tapback plus Copy and More buttons. ↩︎

This Week's Sponsor:

Kolide

Ensure that if a device isn’t secure it can’t access your apps.  It’s Device Trust for Okta.