Notes

Notes is one of the most remarkable comebacks in iOS’ app history. With iOS 9 two years ago, Apple took the popular, but simplistic foundation of Notes – often derided for its faux leather pad and use of Marker Felt – and rethought it as a powerful repository for text, documents, and sketches. Suddenly, Notes was modern and flexible, even for power users. Apple’s greatest accomplishment with the relaunch wasn’t just the willingness to go back to the drawing board – it was the balancing act therein. Notes was entirely new, but it didn’t alienate existing users by becoming more difficult to use. The evolution felt natural; it was a perfect fit for iOS 9 given the focus on mobile productivity.

Notes has continued to grow over the past two years with the additions of private notes and collaboration, further cementing its role as a modern day Evernote made by Apple. Aside from a stint with Bear, I haven’t considered switching to any other note-taking app; Notes does everything I need better than anything else.

Apple isn’t done with Notes, which in iOS 11 is getting its heftiest update since 2015. The strategy is no different: Apple wants Notes to be a one-stop shop for anything digital on iOS. To achieve that, the company has refined and extended several core aspects of the app.

Editing Notes

Apple brought some significant improvements to editing and formatting notes in iOS 11.

The format menu, previously limited to text styles and lists, now features formatting shortcuts, buttons to control list indentation¹⁶, and a new monospaced font option. Like other text styles, the latter applies to entire lines rather than individual words.

Notes features a slightly modified version of the San Francisco font in iOS 11.

The updated menu simplifies common formatting operations in the app, saving steps that would have required a trip to the copy and paste menu or an external keyboard in the past.

Tables are now supported in Notes, although in a simplified fashion compared to advanced tables in iWork. With a new Table button in the Shortcuts Bar (or by hitting ⌥⌘T), you’ll create a default 2x2 table within a note. Tables in Notes have no style options for headers or indexing for columns and rows – they’re not labeled, and you can’t change background colors or add full-width lines like in, say, Pages. These are plain tables to outline something in a few seconds without the complexity of advanced settings.

A table in Notes. You can’t add images or checkboxes to cells in a table.

All you can do is tab through cells with the keyboard or tap one to reveal a “grabber” that you can use to select columns and rows and show a menu to add or delete items.¹⁷ Furthermore, you can hold the grabber to rearrange a selection within a table – it’s intuitive and fun, and it makes me wish other note-taking apps offered similar table creation tools. This more straightforward approach makes sense for the nature of Notes, which doesn’t need the complexity of table controls from iWork.

As part of a major reassessment of the Pencil’s interaction with Notes, iOS 11 also changes how drawings are created in the app. The updated functionality is a step forward that splits Notes’ drawing capabilities in two separate modes.

In addition to full-screen sketches, you can now add inline drawings to a note. Previously, drawing required opening a separate screen to draw and export your creation as an image. That distinction provided users with a standalone, distraction-free sketching area, but it wasn’t great for jotting down something while using the keyboard and holding the Pencil. Switching back and forth between typed text and drawings could get tiresome over time. In iOS 11, live drawing areas can be freely intermixed with text, allowing you to handwrite and draw more quickly without constantly alternating views.

Inline drawings are added in two ways. You can either tap the pencil icon in the toolbar (it’s the same icon used for system-wide markup), or you can enable the Only Draw With Apple Pencil mode in Settings ⇾ Notes, which will automatically initiate drawing when the Pencil touches the screen (your fingers will be used for scrolling).

This happens the first time a Pencil touches the screen in Notes for iOS 11.

When a drawing session begins, the view doesn’t change: you stay in the body of a note, but a horizontal separator appears to delimit text and sketches – the two still can’t overlap.

An inline drawing in Notes for iOS 11.

With inline drawings, there’s no concept of entering and exiting the primary note view: at any point while holding the Pencil, you can tap outside of the delimiters to return to typing mode and vice versa. You can also grab the separators and extend the available drawing area vertically if you need more space within the note.

The inline drawing area has been extended with the dividers.

This isn’t the same freeform approach of OneNote or the kind of inline sketching that I envisioned, but the overall flow is superior to iOS 10 if you spend a lot of time switching between keyboard and Pencil.

If you want a dedicated sketching area, you can still draw with the classic full-screen drawing view that contains all the color choices and tools you’re familiar with.

Full-screen sketches can be created with the Add Sketch button of the + menu, and they stand out from inline drawings because they have a white background in the note. There are no new features for full-screen sketches in iOS 11.

A full-screen sketch is displayed as an attachment to the current note.

How inline drawings have been implemented leads me to believe they’re primarily intended for handwriting rather than artistic purposes. When drawing inline, you’re limited to five colors, the ruler tool isn’t available, and the eraser switches to a tap behavior to delete entire strokes instead of supporting continuous erasing. There’s also a lasso tool that lets you select portions of a drawing to drop it somewhere in a note (or elsewhere in another app on iPad). The message is clear: if you want to create more complex drawings, go full-screen; if you mostly want to handwrite or add basic sketches, use inline drawings.

The focus on handwriting is also hinted at by a new Lines & Grids feature to change the page type that serves as a drawing’s background.

There are six alternative page options for drawing and handwriting sections of a note, from spacious lines to a dense grid view. The page type can be changed on a per-note basis with a button in the share sheet, or you can set a default background in Settings ⇾ Notes ⇾ Lines & Grids. Different backgrounds are applied to all inline drawing sections of a note at once; you can’t have multiple page styles for individual sections in the same note. Lines and grids do not apply to full-screen sketches at all.

Different page backgrounds are a clever solution to get more kinds of users to switch to Notes – whether they’re college students or foremen taking notes at the construction site. Lines and grids are an obvious complement to an inline drawing feature skewed towards handwritten text.

Notes’ page backgrounds don’t grant the same flexibility of utilities such as GoodNotes, but they’re good enough for most people. If you create a tall drawing area and apply a grid background to it, Notes transforms into a notebook not dissimilar from dedicated handwriting apps. Notes’ advantage is that a single note can contain formatted text, a full-screen sketch, image and document attachments, a handwritten section, and even links saved from Safari. To my knowledge, there is no other digital notebook as intuitive and integrated as Notes for iOS 11.

Pencil

Apple is highlighting the Pencil as a companion to multitouch rather than a full UI navigation replacement. There are two other Pencil-related additions in Notes for iOS 11, both of which feel like experiments to understand the accessory’s untapped potential.

The first one is handwriting recognition.¹⁸ Twenty-plus years after Newton, Apple is again trying to recognize handwritten text and make it searchable across apps and the OS. The 2017 version of this feature uses machine learning to see and understand text, with better results than the days of Martin and Martha.

In my tests with elementary, block letter handwriting, Notes performed quite well. Handwritten text (from inline drawings) became searchable inside Notes in a matter of seconds, and I was able to type a query and see the corresponding word highlighted in the note. There was the occasional word that couldn’t be recognized, but I still managed to find the note I was looking for by searching for other words I remembered.¹⁹ Notes can even suggest a title for a note based on handwritten text recognized by iOS. I would have liked to see a ‘Copy as plain text’ option to bulk-extract text from a handwritten block in a note, but that is absent from iOS 11.

I’ve been talking to people with more elegant and elaborate cursive handwriting than mine though, and, according to them, Notes’ recognition leaves much to be desired. Obvious mistakes occur in the recognition process – such as parsing hyphenated words that don’t exist in the dictionary – and the technology hasn’t improved much over the course of the summer during the iOS 11 beta phase. It’s unclear if the system will improve at a rapid pace now that iOS 11 has been released to the public.

Title suggestions based on recognized handwriting.

It’s hard to judge a feature such as handwriting recognition when everyone’s handwriting is peculiar in its own ways, but it seems odd that Apple’s take can still be inadequate while competing apps and services do a (mostly) solid job. It’s important for Apple to get this right if handwriting is to ever become a system-wide Pencil feature or a developer API; it would be a missed opportunity not to lay those building blocks.

The other Pencil-only feature is a shortcut called Instant Notes, which allows you to jump back into note-taking in a couple of seconds.

Instant Notes works by tapping the Pencil on the Lock screen (with the display turned on) to reopen a recent note or create a new one. When the Notes app is launched via this method, only the note you’ve selected to be available for quick access can be viewed without a passcode or Touch ID authentication; attempting to show the Notes sidebar or open other notes will ask you to unlock your device, as you’re still in a special locked mode designed specifically for Notes.

Instant Notes’ behavior is managed by two options in Settings ⇾ Notes. First, you can choose to require a passcode when reopening a note from the Lock screen; shorter times are more secure, but you can set the passcode to last all day or to never come up. Also, you can set Instant Notes to always create a new note, resume the last note created from the Lock screen, or resume the last one viewed in the Notes app. These settings modify the behavior of Control Center’s Notes shortcut too; if you want, you can disable Instant Notes from the Lock screen but retain the Control Center button. If you cancel authentication for Instant Notes on the Lock screen, iOS 11 automatically falls back to creating a new note, preventing access to the rest of the app.

My usage of the Pencil tends to be stationary, casually doodling and jotting down thoughts in handwritten mind maps. I don’t need Instant Notes, but I think it’s a big deal for users who walk around with their iPads a lot and don’t spend most of their time typing on a keyboard attached to the iPad.

Instant Notes is a thoughtful way of bundling Notes’ new features with the Pencil itself. The union of the two can transform the iPad Pro – particularly the 10.5-inch model – into a powerful digital notebook unlike anything else.

Document Scanner

It’s a well-known adage²⁰ that every note-taking app expands until it includes a document scanner. The ability to convert physical documents into PDFs has been democratized by the iPhone’s camera on every iteration of its hardware; businesses have been built on top of digital scanners fitting in our pockets, while bigger companies have assimilated the functionality into their more complex and sprawling services. Notes’ new document scanner falls in the second camp: scanning has been incorporated as a feature of the app that lives alongside other types of attachments.

iOS 11’s Scan Documents button presents a familiar interface: it’s a live camera view with options for color capture (you can save a scan in color, grayscale, black & white, or as a full photo), flash activation, and shooting mode (it can be automatic when a document is recognized, or manual). The scanner’s behavior has been modeled after the hundreds of competing apps already on the App Store: when you point the camera to a document, a yellow rectangle overlay previews the recognized document in real time; wait a few seconds, and, if auto mode is enabled, the picture will be taken automatically, generating a scan.

You can take multiple pictures in a row and they’ll be added as individual scans before they’re saved as a single document in a note. Before you commit and generate a scan out of the pictures you’ve taken, you can apply basic modifications to each one, such as color correction, cropping, and rotation. You can also retake or delete scans during this process.

Notes’ new document scanner.

Once saved, scans become native attachments previewed inline within a note. If you choose the Large Images setting for note attachments (which applies to images, sketches, and links too), scans will be displayed as a large swipeable gallery that lets you scroll through every captured document.²¹ Scanned documents are fully non-destructive: after they’ve been saved, you can still go back to cropping the original picture or choosing color types; the only edits you’ll lose are annotations (which are supported to edit documents with your finger or Apple Pencil). Finally, you can rename scanned documents and export them as a multi-page PDF to Files and other apps.

A scanned document with the Large Images setting in a note (left).

I only have minor complaints about Notes’ document scanner. Documents can only be exported as PDFs, not JPEGs, and small documents aren’t cropped. If you scan items such as business cards or receipts, they’ll be embedded in a standard-size page that adds white space all around the document. I would have preferred to end up with differently sized pages in the same PDF.

On the surface, Notes’ document scanner may appear overly similar to dozens of scanner apps we’ve seen over the years. Apple isn’t reinventing the wheel here – but why would they? The iPhone as a scanner is a proven implementation of the camera that millions of users know how to operate at this point; like Reminders and Podcasts before, the majority of users are better served by a free, default solution that works like the third-party apps they’re already familiar with – which is pretty much the definition of Sherlocking.

Why would Apple build a document scanner into Notes in the first place though? I think it’s very likely that they know people are using third-party scanners to generate PDFs that are later archived in Notes or iCloud Drive. With iOS 11, that becomes a single workflow in Notes. Scanning documents fits Notes’ all-in-one philosophy perfectly. I’d wager the document scanner is going to become one of Notes’ most popular features.

However, it wouldn’t be accurate to describe Apple’s document scanner as an incarnation of Sherlocking without its own merits.

There’s a profound difference between the real-time camera overlays of third-party scanner apps and what Apple has done with Notes. Apple’s document scanner is powered by iOS 11’s advanced Vision APIs, which are used to detect documents (as rectangular shapes) and pass them to the camera to capture a scan.

Vision and Camera technologies can automatically find the edges of a page, correct any skewing of the image, and apply filters for shadow removal and other readability enhancements. At a more practical level, they let documents scanned at odd angles be converted to legible front-facing scans.

Third-party apps often implement angle-independent scanning poorly, or don’t support the feature at all and ask you to hold the iPhone’s camera precisely over a document laying flat on a surface. After trying Apple’s document scanner from multiple angles you’re comfortable with, other scanners will feel primitive and limited. In my tests, Notes’ document scanner has done a remarkable job at identifying edges of documents photographed at 45-60 degree angles; the manual adjustments I had to perform with the crop tool were usually minimal.²²

But wait, there’s more. Notes’ computer vision powers an impressive preview UI used to adjust cropping after a scan has been captured. Instead of forcing you to adjust the crop selection on a “flat scan”, Notes goes back into 3D mode to let you crop in the context of the original scene.

This is unlike anything I’ve seen in other iPhone scanners to date. The animation is a mix of sheer utility, technical prowess, and playfulness – three traits that aptly sum up the addition of a document scanner in Notes.

More Changes

I want to mention three more changes brought to Notes in iOS 11 that greatly enhance the app’s experience.

You can now pin individual notes to the top of a folder. This helps to mark certain items as favorites so they can be more easily found later. To pin a note, swipe right on it until the orange bar fully expands.

Pinned notes from all folders are collected at the top of the All iCloud view, which is a nice way to view all your most important notes in one place.

Shared notes have received some attention in this release too. You can now enable edit notifications for shared notes, so you won’t miss any changes from your collaborators. You can also show color highlights for each user in a shared note; different authors are assigned different colors, so you can see who made certain changes to a note. These settings live in the collaboration menu of a shared note.

Last, Notes’ share extension has been revamped with the ability to navigate folders and search across all notes.

The updated Notes extension in iOS 11.

The redesigned extension is a boon for those who, like me, prefer to append content to existing notes, which are now easier to retrieve from the extension. It’s a welcome improvement over the old extension, and I’ve been using it for MacStories every day.

The Future of Notes

Apple’s multifarious approach to Notes exposes a unifying motif for the app: Notes is a playground for modern iOS technologies tested at scale. Pencil drawings, rich links, collaboration, private notes, and now handwriting recognition and document scanning: these are bigger plans deftly disguised as improvements for Notes the app. In many ways, the future of Notes is the future of core iOS technologies. It’s not a coincidence that some of Apple’s best system functionalities in recent years originally debuted in Notes.

Within this grand strategy, we can discern other recurring themes. It’s evident by now that Apple is on a mission to remake Evernote for the masses, for free. So far, Notes’ lean design and lack of superfluous options have worked in Apple’s favor – in spite of a richer feature set, the app is no more complex than it was two years ago. If anything, it is somewhat surprising that highly requested additions – sub-folder creation on iOS and OCR for text contained in images and documents – still haven’t found their way to the app.

Moreover, Notes is the best expression of the Pencil’s simplicity and deep integration with iOS. The Apple Pencil and Notes have advanced in lockstep, but iOS 11 is showing intriguing signs of Pencil-only functionalities that could break out of the Notes sandbox. Instant Notes from the Lock screen and system-wide markup views are, quite possibly, precursors to a more pervasive Pencil presence on iOS.

Apple is keeping the foot on the gas with Notes. The app’s evolution is instrumental in advancing the platform; its iOS 11 update is an acceleration that matches this year’s pace.