It’s harder than ever to push Apple devices to their limits. Sure, some apps and workflows will do it, but for everyday tasks, Apple silicon has opened a gap between hardware and software that we haven’t seen in a while.
The transformation was gradual with the iPhone and iPad compared to the sudden leap the Mac took with the M1, but the result is the same. There are fewer and fewer apps that push Apple’s chips to the max.
That’s beginning to change with the focus on machine learning and Apple silicon’s Neural Engine. While pundits fret over Apple’s lack of an AI chatbot, developers are building a new class of apps that use local, on-device machine learning to accomplish some pretty amazing feats on all of Apple’s devices.
Great examples of this are the apps by Detail, an Amsterdam-based startup. Detail has two apps: Detail Duo, an iPhone and iPad video production app, and Detail for Mac, which does something similar but with a focus on multi-camera setups more suitable to a desktop environment.
As I explained in my Final Cut Pro for iPad first impressions story last week, I don’t work with much video. However, I’ve been dabbling in video more, and I’ve discovered a story as old as personal computers themselves.
Every hardware advance that creates a huge amount of performance headroom is eventually consumed by the ever-growing demands of apps. That’s just as true with Apple silicon as it was for other chip advances. What seemed like more power than average consumers would ever need quickly becomes a necessity as apps like Detail Duo and Detail push that hardware to its limits.
It’s these sorts of advances that I find incredibly exciting because when they’re coupled with intuitive, well-designed apps, they open up entirely new opportunities for users. For Detail, that means simplifying and democratizing video production that would have been out of reach of most users not that long ago, expanding access to video as a creative outlet.
Before digging into these apps further, though, you should know that my son Finn is on the team building Detail and Detail Duo. That’s one of the reasons I’ve known about and followed these apps for a long time now. I figured that’s context readers should know.
Detail Duo
Detail Duo and Detail serve two very different kinds of users, but both apply machine learning in ways that make it easy to create videos. On iOS and iPadOS, Detail Duo leverages both cameras, person segmentation, and speech recognition, combined with Apple’s Metal frameworks. That allows users to do things like film themselves in the foreground of a vertical video shot with the camera on the backside of an iPhone or iPad. That’s a lot of pieces to assemble continuously in real-time on any device, let alone a passively cooled iPhone or iPad.
In total, Duo features five camera layouts:
and three framing shapes that come in seven colors:
Unless your face is being composited onto the view from the back camera, you can switch the camera sources for each layout. You can also replace a camera source with any of ten different pre-built animations or a pre-recorded video. Plus, all videos can be shot in .5x, 1x, and 3x unless the selfie camera is your only source.
After setting up how you want to shoot your video, just hit record to get started. You can record multiple clips, all of which will appear near the bottom of the screen as separate clips. There’s a play/pause button to review your recording, and you can scrub along the timeline by swiping on it. Instead of moving the playhead, the entire timeline moves across a stationary playhead, which works well but was different from other apps I’ve used, which surprised me at first. Beneath those controls are buttons to split and delete clips, generate captions for your video using machine learning, and change the camera layout, frame shape, and frame color.
What those buttons mean is that you have full control to change your video after you’re finished recording, which is Detail Duo’s most compelling feature. This all works because the app is recording from both cameras and creating separate assets for each element of the video. That allows you to change things up later before sharing your video. It also means you can save individual clips from a project, the entire source videos, or save the project’s .detail file, which is a bundle of the project’s assets.
Detail Duo’s file format is a package complete with audio and video files, a proj.data file that describes the video’s structure in a JSON format, a thumbnail, and transcripts of audio converted to text. It’s a clever structure that users don’t have to deal with, but that makes Detail Duo projects flexible and portable.
Detail Duo was clearly built as an iPhone-first app, which is where I expect most people would be likely to use it. However, the app is also offered on the iPad, where it feels like a blown up version of the iPhone app. Detail Duo would make sense as an iPhone-only app, but as long as it’s also offered on the iPad, its design should be rethought to work better with that platform.
Detail for Mac
Detail offers a variety of Scene layouts that are good for things like presentations and product demos.
Detail for Mac is a little different but draws on many of the same ideas. The primary difference is Detail’s emphasis on multi-cam recording. Camera sources can include:
- Any camera input recognized by your Mac, such as connected DSLR cameras
- An iPhone or iPad camera that supports Continuity Camera
- An iPhone or iPad that uses Detail Camera, a companion iOS and iPadOS app that allows multiple older iPhones and iPads to be connected and used with Detail for Mac wired or wirelessly
- A shared Mac screen or window
- Scenes created in Detail, which I’ll explain in a moment
The sources above are the component parts of what Detail calls Scenes, which can use one or more sources to create a single-view Scene or a Combo Scene that comes in five different layouts. That allows you to mix and match camera sources with screen-sharing sources and even other scenes into one composite video. The result is a flexible system that can be easily adapted for video interviews, tutorials and help documentation, video conferencing apps like Zoom, and more. Moreover, you can set up multiple scenes in advance and switch among them using keyboard shortcuts or Apple’s Shortcuts app.
Like Detail Duo, each Scene you create for a landscape or vertical video is recorded simultaneously as a separate clip. That means that instead of having to perfectly time your Scene switches while recording, you can start with one Scene’s clip, split it, and then swap in the clip from another Scene, taking a lot of the pressure during the recording process.
Detail also supports:
- Insetting your video sources and adding backgrounds in the border around each source
- Video filters
- Text overlays with multiple style options and the ability to place the text in the foreground or background of your video
- Center Stage-style auto-framing of subjects
- Taking still snapshots of a frame of video for use as a thumbnail
- Any audio input source available on your Mac
- AI-generated transcripts exportable in SRT and VTT formats
- Magic Markers that suggest highlights that can be exported as separate clips
- Insights, another AI feature that suggests titles and descriptions for your video
- Importing video from other sources
- Adding markers manually
- Pre-built Scene Packs available from Detail’s website
- Exporting as a standalone video or Final Cut Pro XML file for editing in the Mac version of that app
I’m a big fan of Detail’s automation integrations too. There are keyboard shortcuts for everything. Plus, Detail lets you control your multi-cam setup with Shortcuts, which is something not many apps do. The app’s Shortcuts actions include switching between Scenes or to a previous Scene, taking snapshots of your video, starting, stopping, or toggling recording, and switching scenes at an index.
Overall, I’ve found Detail easier to get started with than most video-creation apps. However, one thing that I think would help get Mac users up and running faster would be to make use of the Mac’s menu better. Many features are behind buttons that only appear on hover or require right clicks, making them hard to discover. The app’s Settings are sparse and unconventional for a Mac app too. These aren’t deal-breaker issues, but they do create friction for new users.
There are a lot of apps for making videos. When I reviewed Final Cut Pro for iPad, I felt like I was using a fancy version of iMovie designed to create traditional movie-style videos with title cards and credits at the end. What I like about Detail is its flexibility to present clips from multiple angles, with adjustable styles and formats that fit perfectly with social media, streaming services, and web-based video.
By taking advantage of Apple Silicon’s Neural Engine, Detail Duo and Detail for Mac put powerful tools in the hands of creators who can mix and match sources and styles on the go or quickly edit something together at their desks with the help of machine learning and AI. It’s a different style of editing that focuses on speed and efficiency while maintaining flexibility and personality, which is perfect for video storytelling in 2023.
Detail Duo is available on the App Store as a free download with certain features available for a subscription that’s $7.99/month or $34.99/year.
Detail for Mac is free to download from the Mac App Store, but pro features, including 1080p and 4K exports, AI transcriptions, and XML exports, require a subscription that is $11.99/month or $69.99/year.
