This Week's Sponsor:

Quip

A supercharged clipboard manager for Apple devices with on-device intelligence, iCloud sync, and text expansion.

Posts tagged with "developer tools"

Swift Assist, Part Deux

At WWDC 2024, I attended a developer tools briefing with Jason Snell, Dan Moren, and John Gruber. Later, I wrote about Swift Assist, an AI-based code generation tool that Apple was working on for Xcode.

That first iteration of Swift Assist caught my eye as promising, but I remember asking at the time whether it could modify multiple files in a project at once and being told it couldn’t. What I saw was rudimentary by 2025’s standards with things like Cursor, but I was glad to see that Apple was working on a generative tool for Xcode users.

In the months that followed, I all but forgot that briefing and story, until a wave of posts asking, “Whatever happened to Swift Assist?” started appearing on social media and blogs. John Gruber and Nick Heer picked up on the thread and came across my story, citing it as evidence that the MIA feature was real but curiously absent from any of 2024’s Xcode betas.

This year, Jason Snell and I had a mini reunion of sorts during another developer tools briefing. This time, it was just the two of us. Among the Xcode features we saw was a much more robust version of Swift Assist that, unlike in 2024, is already part of the Xcode 26 betas. Having been the only one who wrote about the feature last year, I couldn’t let the chance to document what I saw this year slip by.

Read more

Hands-On: How Apple’s New Speech APIs Outpace Whisper for Lightning-Fast Transcription

Late last Tuesday night, after watching F1: The Movie at the Steve Jobs Theater, I was driving back from dropping Federico off at his hotel when I got a text:

Can you pick me up?

It was from my son Finn, who had spent the evening nearby and was stalking me in Find My. Of course, I swung by and picked him up, and we headed back to our hotel in Cupertino.

On the way, Finn filled me in on a new class in Apple’s Speech framework called SpeechAnalyzer and its SpeechTranscriber module. Both the class and module are part of Apple’s OS betas that were released to developers last week at WWDC. My ears perked up immediately when he told me that he’d tested SpeechAnalyzer and SpeechTranscriber and was impressed with how fast and accurate they were.

It’s still early days for these technologies, but I’m here to tell you that their speed alone is a game changer for anyone who uses voice transcription to create text from lectures, podcasts, YouTube videos, and more. That’s something I do multiple times every week for AppStories, NPC, and Unwind, generating transcripts that I upload to YouTube because the site’s built-in transcription isn’t very good.

What’s frustrated me with other tools is how slow they are. Most are built on Whisper, OpenAI’s open source speech-to-text model, which was released in 2022. It’s cheap at under a penny per one million tokens, but isn’t fast, which is frustrating when you’re in the final steps of a YouTube workflow.

An SRT file generated by Yap.

An SRT file generated by Yap.

I asked Finn what it would take to build a command line tool to transcribe video and audio files with SpeechAnalyzer and SpeechTranscriber. He figured it would only take about 10 minutes, and he wasn’t far off. In the end, it took me longer to get around to installing macOS Tahoe after WWDC than it took Finn to build Yap, a simple command line utility that takes audio and video files as input and outputs SRT- and TXT-formatted transcripts.

Yesterday, I finally took the Tahoe plunge and immediately installed Yap. I grabbed the 7GB 4K video version of AppStories episode 441, which is about 34 minutes long, and ran it through Yap. It took just 45 seconds to generate an SRT file. Here’s Yap ripping through nearly 20% of an episode of NPC in 10 seconds:

Replay

Next, I ran the same file through VidCap and MacWhisper, using its V2 Large and V3 Turbo models. Here’s how each app and model did:

App Transcripiton Time
Yap 0:45
MacWhisper (Large V3 Turbo) 1:41
VidCap 1:55
MacWhisper (Large V2) 3:55

All three transcription workflows had similar trouble with last names and words like “AppStories,” which LLMs tend to separate into two words instead of camel casing. That’s easily fixed by running a set of find and replace rules, although I’d love to feed those corrections back into the model itself for future transcriptions.

Once transcribed, a video can be used to generate additional formats like outlines.

Once transcribed, a video can be used to generate additional formats like outlines.

What stood out above all else was Yap’s speed. By harnessing SpeechAnalyzer and SpeechTranscriber on-device, the command line tool tore through the 7GB video file a full 2.2× faster than MacWhisper’s Large V3 Turbo model, with no noticeable difference in transcription quality.

At first blush, the difference between 0:45 and 1:41 may seem insignificant, and it arguably is, but those are the results for just one 34-minute video. Extrapolate that to running Yap against the hours of Apple Developer videos released on YouTube with the help of yt-dlp, and suddenly, you’re talking about a significant amount of time. Like all automation, picking up a 2.2× speed gain one video or audio clip at a time, multiple times each week, adds up quickly.

Whether you’re producing video for YouTube and need subtitles, generating transcripts to summarize lectures at school, or doing something else, SpeechAnalyzer and SpeechTranscriber – available across the iPhone, iPad, Mac, and Vision Pro – mark a significant leap forward in transcription speed without compromising on quality. I fully expect this combination to replace Whisper as the default transcription model for transcription apps on Apple platforms.

To test Apple’s new model, install the macOS Tahoe beta, which currently requires an Apple developer account, and then install Yap from its GitHub page.

Apple Spotlights Four of the Distinguished Swift Student Challenge Winners

Image: Apple.

Image: Apple.

Earlier this year, Apple selected 350 students from around the world as winners of its annual Swift Student Challenge. From that talented pool, Apple picks 50 Distinguished Winners whose projects stand out from the others. Today, Apple highlighted the work of four of them: Taiki Hamamoto, Marina Lee, Luciana Ortiz Nolasco, and Nahom Worku.

Taiki Hamamoto. Image: Apple.

Taiki Hamamoto. Image: Apple.

Taiki Hamamoto built an app playground to teach people about the Hanafuda, a Japanese card game that he discovered many of his friends didn’t know. According to Apple’s press release:

While Hamamoto stayed true to the game’s classic floral iconography, he also added a modern touch to the gameplay experience, incorporating video game concepts like hit points (HP) that resonate with younger generations. SwiftUI’s DragGesture helped him implement dynamic, highly responsive effects like cards tilting and glowing during movement, making the gameplay feel natural and engaging. He’s also experimenting with making Hanafuda Tactics playable on Apple Vision Pro.

Marina Lee. Image: Apple.

Marina Lee. Image: Apple.

Marina Lee, is a computer science student at the University of Southern California. A call from her grandmother who was alerted to evacuate her home because of wildfires in the L.A. area inspired Lee to create EvacuMate to help users prepare an emergency checklist in case of evacuations like her grandmother’s. In addition:

Lee integrated the iPhone camera roll into the app so users can upload copies of important documents, and added the ability to import emergency contacts through their iPhone contacts list. She also included resources on topics like checking air quality levels and assembling a first-aid kit.

Luciana Ortiz Nolasco. Image: Apple.

Luciana Ortiz Nolasco. Image: Apple.

Luciana Ortiz Nolasco built BreakDownCosmic:

a virtual gathering place where users can add upcoming astronomical events around the world to their calendars, earn medals for accomplishing “missions,” and chat with fellow astronomers about what they see.

Ortiz Nolasco who is 15 and from Nuevo León, Mexico will attend WWDC with the other Distinguished Student Winners and plans to continue work on BreakDownCosmic when she returns home with the goal of releasing it on the App Store.

Nahom Worku. Image: Apple.

Nahom Worku. Image: Apple.

Nahom Worku grew up in Ethiopia and Canada and learned to code during the pandemic. Worku’s submission for the Swift Student Challenge app playground, AccessEd, is designed to offer educational resources in places where Internet connectivity doesn’t exist or is spotty.

Built using Apple’s machine learning and AI tools, such as Core ML and the Natural Language framework, the app recommends courses based on a student’s background, creating a truly personalized experience.

Congratulations to all of this year’s Swift Student Challenge winners. I’m always impressed with the projects we’ve learned about through Apple’s press releases and past interviews we’ve done on AppStories. It’s always a pleasure to watch a new generation of kids learn to code and become the developers whose apps I know we’ll cover in coming years on MacStories.

Whisky Shuts Down Project That Enabled Windows Gaming on Mac→

Not long ago, Isaac Marovitz, the developer behind Whisky, the open source WINE front-end that made it easy to play Windows games on a Mac, announced the project had come to an end. Whisky is how Niléane got Cities: Skylines 2 running on an M2 MacBook Air in 2023, and the project was well-regarded in the gaming community for its ease of use. In shutting down the project, Marovitz encouraged Whisky users to move to CrossOver, a paid app by CodeWeavers.

In an interview with Ars Technica’s Kevin Purdy, Marovitz said:

“I am 18, yes, and attending Northeastern University, so it’s always a balancing act between my school work and dev work,” Isaac Marovitz wrote to Ars. The Whisky project has “been more or less in this state for a few months, I posted the notice mostly to clarify and formally announce it,” Marovitz said, having received “a lot of questions” about the project status.

As Purdy explained for Ars Technica, Marovitz also became concerned that his free project threatened CrossOver, and by extension, WINE itself. Last week, CodeWeavers’ CEO wrote about the shutdown, to acknowledge Marovitz’s work and commend his desire to protect the WINE project.

It’s always a shame to see a project as popular and polished as Whisky discontinued. Some gamers may not like that CrossOver is a paid product, but I’m glad that there’s an alternative for those who want it.

To me though, the popularity and fragility of projects like Whisky highlight that a better solution would be for Apple to open its Game Porting Toolkit to users. The Game Porting Toolkit is built on CrossOver’s open source code. However, unlike the CrossOver app sold to gamers, Apple’s Game Porting Toolkit is meant for developers who want to move a game from Windows to Mac. It’s not impossible for gamers to use, but it’s not easy either. I’m not the first to suggest this, and Valve has demonstrated both the technical and commercial viability of such an approach with Proton, but as WWDC approaches, a user-facing Game Porting Toolkit is near the top of my macOS 16 wish list.

Permalink

Is Electron Really That Bad?→

I’ve been thinking about this video by Theo Browne for the past few days, especially in the aftermath of my story about working on the iPad and realizing its best apps are actually web apps.

I think Theo did a great job contextualizing the history of Electron and how we got to this point where the majority of desktop apps are built with it. There are two sections of the video that stood out to me and I want to highlight here. First, this observation – which I strongly agree with – regarding the desktop apps we ended up having thanks to Electron and why we often consider them “buggy”:

There wouldn’t be a ChatGPT desktop app if we didn’t have something like Electron. There wouldn’t be a good Spotify player if we didn’t have something like Electron. There wouldn’t be all of these awesome things we use every day. All these apps… Notion could never have existed without Electron. VS Code and now Cursor could never have existed without Electron. Discord absolutely could never have existed without Electron.

All of these apps are able to exist and be multi-platform and ship and theoretically build greater and greater software as a result of using this technology. That has resulted in some painful side effects, like the companies growing way faster than expected because they can be adopted so easily. So they hire a bunch of engineers who don’t know what they’re doing, and the software falls apart. But if they had somehow magically found a way to do that natively, it would have happened the same exact way.

This has nothing to do with Electron causing the software to be bad and everything to do with the software being so successful that the companies hire too aggressively and then kill their own software in the process.

The second section of the video I want to call out is the part where Theo links to an old thread from the developer of BoltAI, a native SwiftUI app for Mac that went through multiple updates – and a lot of work on the developer’s part – to ensure the app wouldn’t hit 100% CPU usage when simply loading a conversation with ChatGPT. As documented in the thread from late 2023, this is a common issue for the majority of AI clients built with SwiftUI, which is often less efficient than Electron when it comes to rendering real-time chat messages. Ironic.

Theo argues:

You guys need to understand something. You are not better at rendering text than the Chromium team is. These people have spent decades making the world’s fastest method for rendering documents across platforms because the goal was to make Chrome as fast as possible regardless of what machine you’re using it on. Electron is cool because we can build on top of all of the efforts that they put in to make Electron and specifically to make Chromium as effective as it is. The results are effective.

The fact that you can swap out the native layer with SwiftUI with even just a web view, which is like Electron but worse, and the performance is this much better, is hilarious. Also notice there’s a couple more Electron apps he has open here, including Spotify, which is only using less than 3% of his CPU. Electron apps don’t have to be slow. In fact, a lot of the time, a well-written Electron app is actually going to perform better than an equivalently well-written native app because you don’t get to build rendering as effectively as Google does.

Even if you think you made up your mind about Electron years ago, I suggest watching the entire video and considering whether this crusade against more accessible, more frequently updated (and often more performant) desktop software still makes sense in 2025.

Permalink

App Store Vibes→

Bryan Irace has an interesting take on the new generation of developer tools that have lowered the barrier to entry for new developers (and sometimes not even developers) when it comes to creating apps:

Recent criticism of Apple’s AI efforts has been juicy to say the least, but this shouldn’t distract us from continuing to criticize one of Apple’s most deserving targets: App Review. Especially now that there’s a perfectly good AI lens through which to do so.

It’s one thing for Apple’s AI product offerings to be non-competitive. Perhaps even worse is that as Apple stands still, software development is moving forward faster than ever before. Like it or not, LLMs—both through general chat interfaces and purpose-built developer tools—have meaningfully increased the rate at which new software can be produced. And they’ve done so both by making skilled developers more productive while also lowering the bar for less-experienced participants.

And:

I recently built a small iOS app for myself. I can install it on my phone directly from Xcode but it expires after seven days because I’m using a free Apple Developer account. I’m not trying to avoid paying Apple, but there’s enough friction involved in switching to a paid account that I simply haven’t been bothered. And I used to wrangle provisioning profiles for a living! I can’t imagine that I’m alone here, or that others with less tribal iOS development knowledge are going to have a higher tolerance for this. A friend asked me to send the app to them but that’d involve creating a TestFlight group, submitting a build to Apple, waiting for them to approve it, etc. Compare this to simply pushing to Cloudflare or Netlify and automatically having a URL you can send to a friend or share via Twitter. Or using tools like v0 or Replit, where hosting/distribution are already baked in.

Again, this isn’t new—but being able to build this much software this fast is new. App distribution friction has stayed constant while friction in all other stages of software development has largely evaporated. It’s the difference between inconvenient and untenable.

Perhaps “vibe coding” is the extreme version of this concept, but I think there’s something here. Creating small, low-stakes apps for personal projects or that you want to share with a small group of people is, objectively, getting easier. After reading Bryan’s post – which rightfully focuses on the distribution side of apps – I’m also wondering: what happens when the first big service comes along and figures out a way to bypass the App Store altogether (perhaps via the web?) to allow “anyone” to create apps, completely cutting out Apple and its App Review from the process?

In a way, this reminds me of blogging. Those who wanted to have an online writing space 30 years ago had to know some of the basics of hosting and HTML if they wanted to publish something for other people to read. Then Blogger came along and allowed anyone – regardless of their skill level – to be read. What if the same happened to mobile software? Should Apple and Google be ready for this possibility within the next few years?

I could see Google spin up a “Build with Gemini” initiative to let anyone create Android apps without any coding knowledge. I’m also reminded of this old Vision Pro rumor that claimed Apple’s Vision team was exploring the idea of letting people create “apps” with Siri.

If only the person in charge of that team went anywhere, right?

Permalink

Where’s Swift Assist?→

Last June at WWDC, Apple announced Swift Assist, a way to generate Swift code using natural language prompts. However, as Tim Hardwick writes for MacRumors, Swift Assist hasn’t been heard from since then:

Unlike Apple Intelligence, Swift Assist never appeared in beta. Apple hasn’t announced that it’s been delayed or cancelled. The company has since released Xcode 16.3 beta 2, and as Michael Tsai points out, it’s not even mentioned in the release notes.

Meanwhile, developers have moved on, adopting services like Cursor, which does much of what was promised with Swift Assist, if not more. A similar tool built specifically for Swift projects and Apple’s APIs would be a great addition to Xcode, but it’s been nine months, and developers haven’t heard anything more about Swift Assist. Apple owes them an update.

Permalink

On Apple Offering an Abstraction Layer for AI on Its Platforms

Source: Apple.

Source: Apple.

I’ve been thinking about Apple’s position in AI a lot this week, and I keep coming back to this idea: if Apple is making the best consumer-grade computers for AI right now, but Apple Intelligence is failing third-party developers with a lack of AI-related APIs, should the company try something else to make it easier for developers to integrate AI into their apps?

Gus Mueller, creator of Acorn and Retrobatch, has been pondering similar thoughts:

A week or so ago I was grousing to some friends that Apple needs to open up things on the Mac so other LLMs can step in where Siri is failing. In theory we (developers) could do this today, but I would love to see a blessed system where Apple provided APIs to other LLM providers.

Are there security concerns? Yes, of course there are, there always will be. But I would like the choice.

The crux of the issue in my mind is this: Apple has a lot of good ideas, but they don’t have a monopoly on them. I would like some other folks to come in and try their ideas out. I would like things to advance at the pace of the industry, and not Apple’s. Maybe with a blessed system in place, Apple could watch and see how people use LLMs and other generative models (instead of giving us Genmoji that look like something Fisher-Price would make). And maybe open up the existing Apple-only models to developers. There are locally installed image processing models that I would love to take advantage of in my apps.

The idea is a fascinating one: if Apple Intelligence cannot compete with the likes of ChatGPT or Claude for the foreseeable future, but third-party developers are creating apps based on those APIs, is there a scenario in which Apple may regain control of the burgeoning AI app ecosystem by offering their own native bridge to those APIs?

Read more

The M3 Ultra Mac Studio for Local LLMs→

Speaking of the new Mac Studio and Apple making the best computers for AI: this is a terrific overview by Max Weinbach about the new M3 Ultra chip and its real-world performance with various on-device LLMs:

The Mac I’ve been using for the past few days is the Mac Studio with M3 Ultra SoC, 32-core CPU, 80-core GPU, 256GB Unified Memory (192GB usable for VRAM), and 4TB SSD. It’s the fastest computer I have. It is faster in my workflows for even AI than my gaming PC (which will be used for comparisons below; it has an Intel i9 13900K, RTX 5090, 64GB of DDR5, and a 2TB NVMe SSD).

It’s a very technical read, but the comparison between the M3 Ultra and a vanilla (non-optimized) RTX 5090 is mind-blogging to me. According to Weinbach, it all comes down to Apple’s MLX framework:

I’ll keep it brief; the LLM performance is essentially as good as you’ll get for the majority of models. You’ll be able to run better models faster with larger context windows on a Mac Studio or any Mac with Unified Memory than essentially any PC on the market. This is simply the inherent benefit of not only Apple Silicon but Apple’s MLX framework (the reason we can efficiently run the models without preloading KV Cache into memory, as well as generate tokens faster as context windows grow).

In case you’re not familiar, MLX is Apple’s open-source framework that – I’m simplifying – optimizes training and serving models on Apple Silicon’s unified memory architecture. It is a wonderful project with over 1,600 community models available for download.

As Weinbach concludes:

I see one of the best combos any developer can do as: M3 Ultra Mac Studio with an Nvidia 8xH100 rented rack. Hopper and Blackwell are outstanding for servers, M3 Ultra is outstanding for your desk. Different machines for a different use, while it’s fun to compare these for sport, that’s not the reality.⁠⁠

There really is no competition for an AI workstation today. The reality is, the only option is a Mac Studio.

Don’t miss the benchmarks in the story.

Permalink