This Week's Sponsor:

Inoreader

Boost Productivity and Gain Insights with AI-Powered Intelligence Tools

Posts tagged with "M3"

The M3 Ultra Mac Studio for Local LLMs→

Speaking of the new Mac Studio and Apple making the best computers for AI: this is a terrific overview by Max Weinbach about the new M3 Ultra chip and its real-world performance with various on-device LLMs:

The Mac I’ve been using for the past few days is the Mac Studio with M3 Ultra SoC, 32-core CPU, 80-core GPU, 256GB Unified Memory (192GB usable for VRAM), and 4TB SSD. It’s the fastest computer I have. It is faster in my workflows for even AI than my gaming PC (which will be used for comparisons below; it has an Intel i9 13900K, RTX 5090, 64GB of DDR5, and a 2TB NVMe SSD).

It’s a very technical read, but the comparison between the M3 Ultra and a vanilla (non-optimized) RTX 5090 is mind-blogging to me. According to Weinbach, it all comes down to Apple’s MLX framework:

I’ll keep it brief; the LLM performance is essentially as good as you’ll get for the majority of models. You’ll be able to run better models faster with larger context windows on a Mac Studio or any Mac with Unified Memory than essentially any PC on the market. This is simply the inherent benefit of not only Apple Silicon but Apple’s MLX framework (the reason we can efficiently run the models without preloading KV Cache into memory, as well as generate tokens faster as context windows grow).

In case you’re not familiar, MLX is Apple’s open-source framework that – I’m simplifying – optimizes training and serving models on Apple Silicon’s unified memory architecture. It is a wonderful project with over 1,600 community models available for download.

As Weinbach concludes:

I see one of the best combos any developer can do as: M3 Ultra Mac Studio with an Nvidia 8xH100 rented rack. Hopper and Blackwell are outstanding for servers, M3 Ultra is outstanding for your desk. Different machines for a different use, while it’s fun to compare these for sport, that’s not the reality.⁠⁠

There really is no competition for an AI workstation today. The reality is, the only option is a Mac Studio.

Don’t miss the benchmarks in the story.

Permalink

The M3’s Potential to Transform Mac Gaming→

Raymond Wong has an excellent story on Inverse about the Mac and gaming. Wong spoke to multiple Apple representatives about its push to build Macs that can handle the most demanding PC and console games, exploring the impact of Apple silicon on the company’s efforts. In that vein, Doug Brooks, a member of the Mac product marketing team, told Inverse:

Gaming was fundamentally part of the Apple silicon design. Before a chip even exists, gaming is fundamentally incorporated during those early planning stages and then throughout development. I think, big picture, when we design our chips, we really look at building balanced systems that provide great CPU, GPU, and memory performance. Of course, [games] need powerful GPUs, but they need all of those features, and our chips are designed to deliver on that goal. If you look at the chips that go in the latest consoles, they look a lot like that with integrated CPU, GPU, and memory.

That integrated, console-like approach has the added benefit of bringing the iPhone and iPad along for the ride, greatly expanding the potential size of the market for game developers. According to Leland Martin, one of Apple’s software marketing managers:

If you look at the Mac lineup just a few years ago, there was a mix of both integrated and discrete GPUs. That can add complexity when you’re developing games. Because you have multiple different hardware permutations to consider. Today, we’ve effectively eliminated that completely with Apple silicon, creating a unified gaming platform now across iPhone, iPad, and Mac. Once a game is designed for one platform, it’s a straightforward process to bring it to the other two. We’re seeing this play out with games like Resident Evil Village that launched first [on Mac] followed by iPhone and iPad.

With the introduction of the M3 family of chips, Apple’s gaming story continues to evolve by adding hardware-accelerated ray tracing, mesh shaders, and Dynamic Caching, which determines on-the-fly the amount of memory to make available to the M3’s GPU for improved performance. Those chip enhancements are paired with new developer tools designed to make it easier to bring games to the Mac.

There are a lot of variables at play, and whether Apple can compete head-to-head with PC and console games is far from certain. However, what’s clear is that Apple is doing more than at any time in recent memory to make a run at the top end of the videogame market.

Some of the fruits of those efforts are beginning to appear on the App Store. Capcom’s Resident Evil Village debuted on the Mac in the fall of 2022 and more recently on the iPhone and iPad. As Wong notes, Lies of P, one of the top releases of the year was released on the Mac at the same time as other platforms, and Baldur’s Gate 3 was released on Steam for the Mac just a couple of months after its debut on other platforms. Plus, Capcom is back with Resident Evil 4 on every Apple device, and Death Stranding is slated for early next year. That’s a lot of top-notch games.

I’ve been playing many of these titles across an original M1 MacBook Air, M1 Max Mac Studio, and, most recently, on M3 Max MacBook Pro that Apple sent me, and the early results aren’t surprising. The M1 MacBook Air struggles, while the M3 Max MacBook Pro looks stunning. That may not make any Mac the best choice for gaming today, but with the M3, the technology to make it competitive with PCs and consoles is emerging and will inevitably trickle down to more affordable Macs over time.

Whether that happens fast enough and whether Apple can attract the biggest games are just two of many open questions. However, as we head into 2024, I’m encouraged by what I’ve seen so far and plan to share more of my ongoing exploration of Mac gaming in the new year.

Permalink

Jason Snell’s Hands-On with the M3 MacBook Pros and iMac→

Jason Snell of Six Colors got a sneak peek at the new Macs announced at yesterday’s Scary Fast Apple event. A lot of specs were thrown around by the company yesterday, but a software feature called Dynamic Caching really stood out because it’s clear that Apple is doing all it can to squeeze every bit of performance out of its GPUs. Jason’s explanation of how it works is excellent:

There’s also a big new feature Apple is calling Dynamic Caching. Put very simply, Apple’s chip engineers were extremely motivated to eke out even more performance from their graphics subsystem—and found that the way memory was traditionally allocated was inefficient. Memory is usually allocated to different threads at compile time, meaning that some threads allocate a larger amount of memory in order to handle peak need, while other threads might choose a smaller amount of memory but risk a bottleneck.

The M3’s graphics system dynamically allocates the memory per thread in a way that’s completely transparent to software developers. Apps don’t need to be rewritten to take advantage of the new system, which Apple says makes some huge gains by wringing a lot of memory efficiency out of the system. Memory that was previously reserved for a specific thread can be given to a different thread instead. A thread that’s in a bottleneck can be given more space. It’s all to the goal of increasing overall throughput.

The fact that these improvements come ‘for free,’ meaning developers don’t have to change their apps or games to take advantage of Dynamic Caching, is at least as important as the efficiency gains enabled by the technology. Especially when it comes to things like videogames, the more Apple can do to make it easy for developers to take advantage of Apple silicon Macs, the better.

Jason also got some hands-on time with the new MacBook Pros, including the new Space Black model:

I got my greasy monkey paws on a Space Black laptop and can report that Apple’s as good as its word in the sense that it seems generally more resistant to fingerprints and other smudges.

But I don’t want to exaggerate this feature: you can still see fingerprints. They just aren’t as prominent. This is a progressive improvement over something like the Midnight M2 MacBook Air, but it’s not a cure-all.

Despite its name, Jason reports that Space Black is more gray than black, but it’s still a noticeable shift from Space Gray.

With the details of the new Macs dissected, it’s going to be interesting to see how the M3 MacBook Pro’s latest CPU and GPU configurations perform relative to the M2 models that were released at the beginning of the year. As Jason also points out, the benchmarks we see from the new laptops and the M3 iMac should give us a good idea of how M3 MacBook Airs, Mac minis, and Mac Studios will perform when it’s their turn to be updated.

Permalink

Apple’s October 2023 Scary Fast Event: By the Numbers

Today’s Scary Fast online Apple event was packed with facts, figures, and statistics throughout the presentation and elsewhere. We’ve pulled together the highlights.

M3, M3 Pro, and M3 Max Chips

  • These are the first chips built on a 3-nanometer process.
  • This process can fit up to 2 million transistors in the cross-section of a human hair.
  • The M3 architecture grants up to 2.5x faster performance than the M1 generation.
  • The M3 CPU’s performance cores are 30% faster than M1 and 15% faster than M2; the efficiency cores are 50% faster than M1.
  • The Neural Engine is faster and more efficient in M3 as well. Specifically, it’s 60% faster than M1 and 15% faster than M2.
  • The M3 Max chip is up to 80% faster than the M1 Max.
  • The M3 has 25 billion transistors, while the M3 Pro has 37 billion, and the M3 Max has 92 billion.

MacBook Pro

  • The M3 Max MacBook Pro supports up to 128 GB of unified memory and 8 TB of storage, with a maximum 16-core CPU and 40-core GPU.
  • The M3 MacBook Pro can run for up to 22 hours on one charge, playing movies using the Apple TV app.
  • The new MacBook Pros are up to 11x faster than the last Intel-based models.

iMac

  • The M3 iMac is 2x faster than the M1 model, 2.5x faster than the 27-inch Intel model, and 4x faster than the 21.5-inch Intel-based iMac.
  • The M3 iMac has a 4.5K Retina display and features a 6-speaker sound system.
  • Apple offers up to 24 GB of unified memory and 2 TB of storage in the M3 iMac.

You can follow all of our October 2023 Apple event coverage through our October 2023 Apple event hub or subscribe to the dedicated October 2023 Apple event RSS feed.

Apple Introduces the New MacBook Pro in Three M3 Chip Configurations

Source: Apple.

Source: Apple.

One of the things I’ve enjoyed about the rollout of Apple silicon Macs is that the old rules don’t apply, and the new ones are still being written. The cadence of releases is still settling in, and today, in the face of speculation that Apple was struggling to release M3 Macs, Apple made it clear that not one, but three 3 nanometer process-based chips are ready to ship. Along with the M3 iMac, the company refreshed its entire lineup of MacBook Pros, computers that gained the M2 chip less than a year ago.

Read more