Jul 6, 2024

Comparing Oranges to Apple

Forget MacBooks, the 'Copilot+ PCs' played themselves, quite literally

Following Tom Warren's review of Microsoft's own Surface Laptop a couple weeks ago, Joanna Nelius went ahead and compared a bunch of 'Copilot+ PCs' using Qualcomm's new Snapdragon X chips. The results seem fine – in particular compared to Intel/AMD chips and/or previous stabs at Windows running on ARM – but this entire launch was positioned, at least by Microsoft and Qualcomm, to be about ending the reign of Apple's MacBooks. And again, they simply don't seem to do that. The Verge almost wants to go out of their way not to call that out plainly, but it's right there in the results.

Among the laptops we tested, Apple’s M3 chips still lead the pack in single-core performance, but Qualcomm’s higher-end X Elite chips are a touch faster in single-core workloads than the M2 Max chip in the early 2023 MacBook Pro — between 2 and 3 percent in our tests. They are also up to 24 percent faster than the performance cores in Intel’s Core Ultra 7 155H processor and up to 17 percent faster than the AMD Ryzen 7 8845HS cores. The bottom-tier X Elite and X Plus are the slowest, but they still push out impressive scores and pull ahead — albeit barely — of most of the Intel-based laptops in the table below.

The M2 Max chip was released in January 2023. It's now July 2024. Qualcomm's chip is a "touch faster" than a chip Apple released 18 months ago. Not only has Apple subsequently moved on to the M3 series – they already have the M4 chip live, albeit in the iPad Pro, but it will be coming to the MacBook line soon enough. Apple no longer sells any MacBooks with any M2 Max chips, so I'm not sure why we're even making that comparison (I guess to normalize the CPU cores for comparison?). Qualcomm has slightly beat a machine Apple released a year and a half ago and no longer sells because they sell newer, faster machines. Machines which remain faster than Qualcomm's despite being nearly a year older in the market themselves.

The Snapdragon chips really shine in multicore benchmarks, overtaking all the other CPUs aside from Apple’s M2 Max and M3 Max. The M3 chip in the MacBook Air only has eight cores compared to the 12 or 10 cores of the Snapdragon chips, so it makes sense why it fell behind. The 16-core M3 Max far outpaces the rest of the field, and the 12-core M2 Max is slightly faster in Cinebench 2024 multicore than the fastest X Elite chips.

The Snapdragon chips "shine" – unless compared to Apple's more powerful chips, one of which is 18 months old, the other of which is 9 months old.

Qualcomm's chips do squeak out a win in multicore (while again, losing in single-core performance, which is more important for most tasks) against the standard M3 chip, but as Nelius notes, that's because the Snapdragon chips have two or four more cores than that chip. Apple's M2 Max chips, which have the same amount of cores as these Snapdragons, but again, are 18-months old, still beat the new Qualcomm chips in multicore (while being similar or better in single-core). The M3 Max, which has four more cores than the top of the line Snapdragons, unsurprisingly still blows everything away, despite being last year's chip.

The above sort of hits upon the funny business Microsoft and Qualcomm were trying to pull in their positioning here. They kept touting these new 'Copilot+ PCs' as being faster than the MacBook Air, which is a fan-less computer that runs a CPU with fewer cores as a result. Not stated was that this was only in multicore performance, which requires more power (and thus, fans to cool down the chips outputting that power). In single-core processing, again, likely the more important proxy for day-to-day use, even the Air still beats the new Qualcomm chips.

Meanwhile, Apple's actually comparable chips – including the 18-month old M2 Max – beat the Qualcomm chips in multicore. And the actual chip you can buy in a MacBook right now, the M3 Max, destroys Qualcomm's chips across the board. And an M4 Pro/Max chip is undoubtedly coming shortly, with the M4 chip already in the wild.

Beyond all that, the takeaway is that these machines have pretty poor integrated GPUs and as such, are not good for gaming, one of the key areas where PCs have long had an advantage over Apple as well (though Apple's integrated GPUs seem far superior now). And the emulation, despite also being touted to no end, is actually still pretty bad. Better than in previous versions of Windows for ARM, but only because that performance was absolutely terrible. This is just bad. Bad enough that many software makers are blocking their apps from running on these devices until they can get native versions released.

But wait, perhaps the most highly touted features of these machines are battery life. So how are things looking there? Not great, Microsoft Bob:

Microsoft claims that its Copilot Plus PCs with Qualcomm’s Snapdragon X Elite processor will offer “20 percent more battery life than the latest MacBook Air 15-inch.” When I tested the latest 13- and 15-inch MacBook Airs, they lasted about 18 hours on a charge when I used them as I normally would during a regular week, with the display brightness set as close to 200 nits as possible.

None of the Snapdragon laptops’ batteries lasted 18 hours like the M3 Air, but most weren’t far behind, averaging 14 to 16 hours. That’s still a lot longer than most of the Intel- and AMD-based laptops I tested, with the exception of the Intel-based Samsung Galaxy Book4 Ultra, which got near identical battery life to the Snapdragon-based Galaxy Book4 Edge, at over 14 hours each.

Check my math here, but machines with a few hours less battery life doesn't seem like "20 percent more" battery life. Yes, the battery life is better than Intel and AMD laptops, but that bar is so low it's basically on the ground.

But, but, but AI!

Yes, these are Copilot Plus PCs. Yes, they run a bunch of AI stuff. But my colleagues and I have yet to figure out a reliable method to test relative NPU performance in a meaningful way. The Copilot Plus AI features, with the possible exceptions of Studio Effects and Live Captions translations, currently feel more like gimmicks than useful apps most people can incorporate into their day-to-day life, and Microsoft doesn’t plan to release Recall, its most-hyped AI app, until it addresses security concerns.

Sounds great. Some real selling points right there. The MacBook still looks awfully alive.

🍎

Previously, on the race to take on the MacBook...