> Isn't there a single GPU benchmark that actually does the same work so that comparisons can actually be made?

Apple does not support OpenGL or Vulkan, only Metal, and most app devs have better things to do than rewriting code for the Mac.

The recommended way of gaming on a mac is to use emulation by emulating whatever the game is used with a metal wrapper.

IMO the claim that these benchmarks aren't fair is naive. I don't care about how good the hardware is, but about what performance I get. If I get poor performance because the software, drivers, etc. are poor, I want to know that.

> I don't care about how good the hardware is, but about what performance I get.

Of course, I'm not at all saying that you're wrong to want that, or that these benchmarks don't show what you're interested in.

However this and many other articles are using these benchmarks to derive comparative hardware performance, which is simply wrong to do. That's what I'm criticizing.

Since you can't really buy apple cpus or gpus and put them in a PC, for benchmarking... today at least one can't compare apple's hardware against intel hardware.

What one can compare, is the performance of the "Apple platform" and the "PC platform" at similar price points, power budgets, features. This is more meaningful for most people, which mostly care about how fast can the computer do X (whether it uses the CPU, GPU, or some other chip, most people don't care).

Is this a serious comment? Alwrapper doesn't use emulation. An API wrapper simply translates calls to the appropriate API. But more to the point, If the benchmark isn't properly ported to the platform it's testing then the benchmark is BROKEN. It isn't naive to expect them to do their job right.

> Is this a serious comment? Alwrapper doesn't use emulation. An API wrapper simply translates calls to the appropriate API. B

QEMU simply translates calls from one hardware API to another hardware API, so it isn't an emulator either right?

It depends on your perspective, yours is a subjective perspective that you care your experience only, which makes sense for a consumer to know what to buy. (But then everyone’s subjective bias has different weights so to speak.)

But from a technological perspective, the logical way to test is to eliminate as much variable as possible and really compare the hardware compatibility. Using software that’s not optimized for the hardware is not an objective test of the hardware.

There are many ways to try to conduct as objective a test as possible. In that criteria, I found that only those review from Anandtech is up to standard. Article like this is more like click bait.

Now I’m not an expert, but if I were to compare the performance somewhat objectively, I might start with TensorFlow where Apple has releases a metal backend for it to run. Then may be write some naive kernel in Julia using CUDA and the pre release metal library (It might not be fair, but that’s where I would start given what I know.)

Can't you just install Linux on both machines?

Linux for Macs with M1 architecture is still in development. There's a team that's currently reverse engineering the M1 GPU to create linux drivers for it ... but until that's finished, unfortunately no, you can't just install Linux on both systems :/

Using the in-progress Asahi Linux to benchmark the hardware could be justified as hampering the hardware even more than MacOS's limited support for mainstream graphics APIs.

The GPU is not supported yet in Asahi Linux.

Probably a fairer comparison would be something that runs natively using vulkan or metal on both Apple's M1 and X86/Nvidia that isn't heavily optimized for just one of those. There aren't that many modern games that you could reach for here. But probably something that can target both vulkan and metal would be what you'd want. The upcoming x-plane v12 is something I might like a mac studio for.

What's interesting here is that is that we're haggling over comparing something that is an SOC design with fairly modest energy usage and cooling with a video card and cpu (presumably) that need a lot of power and cooling. Imagine Apple putting a few of these CPUs in the mac pro next year (or whenever they release that).

Of course these things are pricey. 5K for the 64 core model, ouch. I think they are going to be making lots of money selling these.

As for intel, amd, and nvidia. They each need to get their act together on the SOC front. They are not for crippled laptops anymore but for high end work stations as well. Nvidia should re-assess what they want now that their arm acquisition is off the table. IMHO they should get rid of amd/intel as the go-to cpu for their gpu and just build their own SOC and target it at gaming machines, workstations, etc. There's no good reason to stick with x86 anymore either. License ARM, or maybe hop on the Risv V bandwagon.

Nvidia has been making socs targeted at gaming for years now, they are in the Nintendo switch. They have a lineup of chips with arm cores and gpu cores available now but they are targeting industrial applications.

I know. That's exactly the point. It's not a flagship product for them but a watered down thing they aim at low cost devices. Apple charges 5K for a their 64 core Mac Studio. It's the fastest thing they have right now and it's an SOC. The fact that it even comes close to Nvidia's latest and greatest shows that this is the right thing to do from a design point of view.

> Isn't there a single GPU benchmark that actually does the same work so that comparisons can actually be made?

There's IndigoBench, but some people will complain about it using OpenCL (which still works great in spite of Apple's efforts to kill their own creation to the detriment of the whole GPU compute industry) instead of Metal.

Would be interested in benchmarks for this using the new M1 Ultra: https://www.indigorenderer.com/indigobench

Disclosure: worked on Indigo as 1/2 engine devs for 9 years.

>Isn't there a single GPU benchmark that actually does the same work so that comparisons can actually be made?

Octane Render runs on Metal and CUDA so can at least compare compute aspects of GPU.

Presumably because it's a real life workload that is more relevant to consumers and easier to understand.

Sure, if the consumer wants to run Shadow of the Tomb Raider, that's true.

But benchmarks where one hardware has to do much more work than the other is obviously not fit to determine the actual performance differences between hardware; and that is exactly what the article we're commenting on is doing.

Everyone understands the point you're making, it's just a pointless point.

People aren't concerned about the hardware itself. They're concerned about what it can do. If circumstances mean that your powerful hardware can't be utilized, then the powerful hardware is just extra cost and no benefit.

Tom's Hardware. ;-)

World of Warcraft supports M1 and Metal. I'm not sure whether it's good as a benchmark.

And to pile on, the games are tuned to run well on Nvidia hardware. They haven't even seen this new GPU yet or even care to optimize for it. Sometimes even small changes can make big differences.

You often have to choose between fair or relevant benchmarks.