Posts tagged with "iOS 27.3"

Notes from Apple’s Siri AI Labs: Schemas, Tools, and Skills

Earlier this week, I attended Apple’s online Apple Intelligence Group Lab, much of which focused on Siri AI. Much of the ground covered is available in Apple’s documentation, but the presentation included additional detail, which is worth highlighting, although I encourage you to watch the two Apple Intelligence Group Labs for more context and details.

The big-picture change that Federico explained earlier this week is that Apple is greatly expanding the actions available to Siri. Today, Siri AI’s natural-language abilities are limited to tightly defined domains like calendar, reminders, and messages. If a developer’s app doesn’t fit a schema (or hasn’t implemented an App Shortcut), Siri won’t be able to respond to a user.

Apple isn’t dispensing with schemas, but it’s loosening the reins, giving developers far wider latitude than before. Apple noted during the lab that schemas are still developers’ best bet for getting Siri to respond because Apple’s already done the work to ensure they work.

What’s changing is that Siri will be able to directly access developers’ App Intents that have been defined as tools. That’s a big change from today where a developer’s main access to Siri is through App Shortcuts, using a defined phrase that names the app. Instead, if a developer marks their App Intents as tools, Siri will be able to access those App Intents directly when a user makes a natural-language request, much like how MCP works with tools like Codex and Claude Code.

Other interesting tidbits about tools mentioned include that they’ll use Private Cloud Compute to process the request and that apps are limited to 40 tools.

Skills are a third brand-new way to access Siri that’s designed for more complex, multistep actions that can chain up to 32 tool calls. Like the skills you may be familiar with from Codex and Claude Code, Siri’s skills are Markdown instructions with YAML front matter that explain what Siri should do with certain requests. Xcode 27.3 beta also includes coding-agent skills that help developers add Siri support to their apps and test them.

Finally, beginning with iOS 27.3, which will ship in January if history is any indicator, developers will be able to customize the look of snippets of information from their apps similar to how Apple’s own apps’ content can look in Siri. As for tools and skills, those will be available to ship to the App Store in the spring.


“Introducing Adaptive Dark Mode”→

As part of the 27.3 betas released yesterday, Apple also introduced a new Safari feature:

We’re excited to announce Adaptive Dark Mode in Safari 27.3 beta. When enabled by a user, Safari adapts light websites and presents them as dark.

In the spring of 2019, Safari 12.1 became the first browser to support dark mode on the web. A lot of people love using dark mode, but not every website has a dark version. Almost 40% of web page loads are from users in dark mode, and yet only 10% of page loads support it. If you love dark mode, you’ve probably hit a site that felt particularly bright. Adaptive Dark Mode is designed to close that gap, providing a consistent dark experience across the web.

If this sounds familiar, it’s because so many third-party Safari extensions for this very task already exist, including the very popular (and a MacStories favorite) Noir. The difference, of course, is that Adaptive Dark Mode is built deep into the WebKit engine itself now, ships for free in Safari, and offers greater speed and performance since it’s not limited by an extension. It sounds like Apple thought of a lot of details here:

At the core of Adaptive Dark Mode is the transformation of color. Sometimes colors are simply decorative, while other times they have semantic meaning. Adaptive Dark Mode determines when to invert colors so elements remain visible or when to leave colors unchanged.

While creating Adaptive Dark Mode, we developed a color matrix to invert the luminosity of colors while preserving hue. It’s used to make light backgrounds dark, and dark text light. When transforming text color, Adaptive Dark Mode aims for sufficient contrast as defined by the WCAG 4.5:1 contrast ratio. Light CSS gradients are also transformed to dark.

And:

It takes the same care with images. Most are left unchanged, preserving their integrity. But many icons and logos can lose contrast when the background color behind them is inverted. So Adaptive Dark Mode inspects SVG images and transforms colors when needed. For bitmap images, it analyzes the pixel histograms and inverts simple black-and-white logos.

Noir and similar extensions had a great run. Between this, the MCP server added earlier this year, and vertical tabs in 27.3, it’s time for me to go back to Safari everywhere.

Permalink

Siri AI’s Evolution Is Both Surprising and Obvious

Siri AI in iOS 27.3 beta.

Siri AI in iOS 27.3 beta.

As part of the release of the 27.3 developer betas yesterday, Apple announced a substantial expansion of the capabilities of Siri AI. As you recall from my iOS 27 review two months ago, Siri AI launched with support for a limited subset of app domains based on schemas that Apple specifically optimized for the first version of Siri AI. In a major reversal for a mid-cycle update to Siri AI, that limitation is entirely going away: the updated Siri AI will instead work with any kind of app that supports any custom App Intent – as long as the developers do the work of modernizing their App Intents and integrating with Siri. Effectively, though, any app can work with Siri AI now, and Apple calls this the Siri AI Developer Preview.

Read more