granted, wccftech is a grain of salt kind of site. To wit, here's another article....from the same site!

https://wccftech.com/m2-ultra-only-10-percent-slower-than-rt...

The complete lack of any power numbers is extremely telling.

As if that matters on a desktop?

It matters when comparing performance numbers in general?

Depending on the usage, it why and efficiency is impressive on laptops, but not so important on the desktop.

Performance per TDP is always relevant. Sure individuals may choose they just want whatever to maximize a metric, but that’s a different decision entirely. Like comparing a motorcycle and a truck without mentioning all the things different between the two.

TDP is the wrong measure here*. i9-13900K and i7-13700K both have 125W TDP with 253W PL2 specs. By that calculation, the 13900K is more performant per TDP, so is "better" for efficiency.

Computation per kWh (or rate of computation per kW) is the right efficiency metric, not TDP (thermal design power).

* - or at a minimum isn't precisely the right measure here

If you have a workload that pushes the 13700K and 13900K to their turbo limits (and have ensured that those limits are actually configured to be the same), then you really will find the 13900K getting more work done for the same power and energy. That's how the extra cores help in power-limited scenarios.

It does when you multiply it across millions of CPUs. Saving 100 watts across 3 million CPUs (300 megawatts) is enough to shut down the equivalent of power plants.

I'm all for efficiency but if we are to progress as a civilization we need to find a way to not have power hold us back. We can get to a point where we have nearly unlimited power with minimal impact on the environment around us. That needs to happen as soon as possible.

There are an estimated 3 billion people in the world who still rely on solid fuel (wood, coal, peat moss) to cook food and generate heat [1]. We are extremely far away from being able to use 100% electricity to meet two basic needs for the whole population. From that perspective power-plant-scale computing is an incredible luxury.

[1] https://www.jstor.org/stable/pdf/resrep27829.23.pdf

That demand will only increase, which further emphasizes my point. It can't be a luxury.

> That needs to happen as soon as possible.

But it won't happen before all M2/3 Macs will be obsolete. Having goals like that is great but until we reach that point efficiency is still important.

I don't like to listen to the sound of fans constantly running ...

My M1 Pro is about 33% as fast as a box with a 3060 in it on PyTorch and obviously it is way more efficient and doesn't require a special power supply, which is just taken for granted as something you have to do with a PC.

The best case this article can make is that if you need to play the latest game or do intense ML stuff you probably want NVIDIA, but that's the same as it ever was.

It also costs a lot more. For the money you can get a laptop with a 4080m and reduce its power limit and probably get the same or better efficiency, but with room to the top if you need it.

And I like it more, so I am willing to pay more. The implicit notion that I’m getting the same thing for less just isn’t true.

You are getting better performance potential and equal or better efficiency at comparable wattage, for a better price. Of course you will not get it the shape of an Apple laptop. We were talking about how Apple chips supposedly have superior efficiency, but at least in terms of GPU that is not true at all.

Keep making excuses. It's still slower.

My audi S5 is slower than a GT3 Touring car.

It's just excuses.

Absolute best of class? What?

A 4080 is best of class?

And second of all, for the price it should be compared to the 4090, which absolutely demolishes it.

> that consumes massively more wattage

power consumption doesn't scale linearly with performances

the absolute best of class NVIDIA discrete GPU offering could possibly outperform the Apple GPU at the same power level

Or, to put it in another way, to recover that remaining 50% of performances (2x) the increase in power consumption would be exponential (a lot more than 2x, like 10x)

They said it's not faster than the desktop parts, which is almost a false statement since it actually is faster at multithreaded than both base desktop SKUs. And then they switch to providing a specific percentage when comparing the the 4080 (also based on an deceptively narrow comparison, one their other articles counter) for dramatic effect.

I don't think it was ever meant to be the most informative article: it seems written to serve the contrarians because that's profitable from a readership perspective for a publication like this.

The absolute "best of class" Nvidia card is currently the 4090, which can be up to 30% faster than the 4080 in scaling workloads.

What would be more interesting is to see how Nvidia's laptop cards fare here though - they're constrained to much lower wattage (80-120w) and would make for a much fairer fight against the ~200w M2 Ultra.

> What would be more interesting is to see how Nvidia's laptop cards fare here though - they're constrained to much lower wattage (80-120w) and would make for a much fairer fight against the ~200w M2 Ultra.

Doesn't look like Apple offers Ultra in a laptop - just the Basic, Pro, and Max.

It's interesting. I assumed at least their power consumption should have been much better/lower than an avg orthodox desktop setup of current days.

It's not unreasonably high for a TDP, and the idle consumption of ARM is de-facto lower than power an x86 package.

That being said, it's pretty obvious that Apple's mobile-style solution isn't really working out on the desktop side of things. The new iMac feels starkly pedestrian compared to the old ones, and the Mac Mini/Studio are both neat but not unprecedented. The M2 Ultra represents a lot of engineering effort going into flipping that status quo, but its still slipping behind by a considerable margin. Don't forget that a second "Ultra" style SOC with 4x M1 Maxes was supposedly cancelled for drawing too much power and being too hot. It's just not effective or efficient to force that much silicon that close together.

A decent 4090 GPU alone is over $3000 in Australia - that’s half the price of a Mac Studio with Ultra - just for the GPU.

Then you’d be looking for a 24 core CPU, 64GB RAM, 1TB PCIe 5 SSD, mainboard with 6x thunderbolt ports, a silent cooling setup, high quality case that is both small and all the gear while running cool. and if you’re stuck with MS Windows - an Operating System.

Yeah, you gotta pay a premium if you want the highest performance on the market. If you're just looking to match the GPU performance of the M2 Ultra though, there are several gaming laptops that will run circles around it in Blender. Many are cheaper than the base model Mac Studio.

Most consumers don't care about wattage on a desktop, and when you compare the prices they are similar and it loses.

I wonder how long you have to run the Apple chip to save enough on electricity to make the price difference worthwhile? Does it only drop to one human lifetime in places where energy cost is insanely high?

Many people do care - many Americans have subsidized low power but that’s not globally true and even if your power is free a hot CPU means you’re listening to fan noise and heating up the room. In a cold winter that’s not bad but I knew people who had to do things like leave their office doors open for cooling.

the people you describe aren't buying $7k desktops

They certainly are. Being able to pay for the extra power doesn’t mean you like fan noise, not to mention the people who are doing things like video production in a trailer where overheating is a concern. I used to support scientists who didn’t appreciate the heat 4 workstations made in their shared offices, either.

A dedicated GPU with more die area and its own large fan will run both cooler and quieter when its power limit is reduced to match the performance of the Apple chip.

More watts in = more heat out.

It doesn’t matter if the cooling solution gets the chops’s surface temperature lower if it’s heating the room twice as fast.

Chip surface temperature is not a useful metric for this purpose.

The die area matters in terms of how much performance you can still get out of it when lowering the power to match up with the (not actually) more efficient integrated chip.

If Apple clocked their GPU to match the performance of a comparable dedicated chip, it would be just as inefficient, noisy and hot. Except they can not do even do that. They turned a limitation of the design into a supposed feature.

>many Americans have subsidized low power

Source?

You have confused HN population with general US population

Kind of. I did a quick comparison this morning between a MacPro and a similar high-end Dell workstation and the prices are similar.

https://twitter.com/0xDEADBEEFCAFE/status/166747612998729728...

I think a useful way of thinking about this is it has something like $300-400 worth of discrete GPU equivalent performance assuming that the article's claims are correct (not exactly a given...) as it will get you something similar to a 3060 ti or 4060 ti.

Wait, a $7000 computer doesn’t come with a discrete GPU?

It's not just that it doesn't come with a discrete GPU. Apple Silicon doesn't support discrete GPUs to begin with.

As far as I understand it—and this is just from watching Apple's presentations on the architecture—the lack of a discrete GPU is a big part of how the Apple Silicon machines achieve good performance per watt.

Instead of having discrete RAM or a discrete GPU with its own VRAM, all of the RAM is accessible to the CPU and and the GPU in a unified memory architecture. On the M2 Ultra, this allows for 800 GB/s of memory bandwidth, and also eliminates a lot of the need to copy data from RAM to VRAM, as both the GPU and the CPU can access the same memory. In return, this allows the GPU to match the performance of discrete GPUs that have a lot more cores.

Of course, the big downside is that you can't expand the RAM or install a beefier GPU. It's all baked in to the logic board.

> and also eliminates a lot of the need to copy data from RAM to VRAM, as both the GPU and the CPU can access the same memory. In return, this allows the GPU to match the performance of discrete GPUs that have a lot more cores.

Plenty of PC hardware reviewers have done sensitivity analysis experiments to see how discrete GPU performance is affected by running with a slower or narrower PCIe link. The consensus is usually that GPUs connected by PCIe have more than sufficient bandwidth, and cutting it in half only affects gaming framerates by a few percent. Tighter coupling between CPU and GPU can plausibly have a bigger impact for some GPU compute workloads, but for traditional 3D graphics it doesn't help performance much.

You can generally ignore FPS numbers as irrelevant outside of extremes, what people notice is stutter and that’s where PCIe link bandwidth matters.

I think it's more accurate to say that PCIe link bandwidth matters when your VRAM capacity starts to become insufficient, a problem that Apple sidesteps. The key advantage of Apple's unified memory strategy isn't fast communication between CPU and GPU, but in granting the GPU access to a high-capacity memory pool (with some side benefits of giving the CPU access to GPU-like memory bandwidth).

Measured as one percent lows and again, most GPUs under most Conditions simply don't saturate PCIe enough for that to make a difference. Der8auer and GamersNexus on youtube have plenty of examples.

1% lows in a given benchmark isn’t really measuring stutter in practice, what you want is percentage of frames that don’t finish in time.

If once a minute you miss 4 frames in a row that’s noticeable even if everything else is rock solid. The thing is people adjust their resolution/settings to reach acceptable FPS, thus it’s fairly GPU independent. What matters is rendering volatility as assets are loaded etc.

That metric is irrelevant now that we have adaptive refresh rates. You don't really "miss" a frame anymore. 1% low is now the correct metric.

If the pipeline stalls for 70ms it’s a problem even if in theory you don’t “miss” a frame you still aren’t getting new frames until rendering finishes.

All adaptive refresh rates do is avoiding unwanted delays after the frame finishes.

That's not how it works. A 70ms pipeline stall means a ~90ms frame which means that your 1% framerate low is going to be reduced.

Adaptive refresh rate means that only frame rate matters. You can't miss a frame, you can only have a frame take longer, and that's measured perfectly well by 1% or 0.1% low frame rates.

Also I don't see why an M series CPU is going to be any better at feeding the (asynchrous, multiple frames in flight) GPU rendering pipeline than a modern x86 CPU, when both are just as fast. macOS is also pretty bad at realtime scheduling for heavy workloads compared to modern Linux and Windows.

Yes occasional 90ms stall reduces 1% frame rates, but less than how annoying it is.

A 10 minute test at 120 FPS is 10 * 60 * 120 ~= 72,000 frames. Your 1% lows is an average of ~720 frames but so what if of 300 them are at 60 FPS you’re going to have trouble noticing.

While if you’re almost rock solid at 120FPS a dozen 90ms stalls it’s really obvious that something is wrong but the calculated 1% low’s actually look better.

The comparison isn’t average FPS or even 1% lows where dedicated graphics cars have an advantage, the comparison is what happens when things go very wrong. And it’s in that very specific case when they may have an advantage.

> While if you’re almost rock solid at 120FPS a dozen 70ms stalls it’s really obvious that something is wrong but the calculated 1% low’s may actually look better.

I think this is an instance of the coordinated omission problem: you cannot simply measure the latency of the frames that were completed, but instead have to consider the frames that should have been delivered during a stall. Looking at frame time percentiles means a stall only penalizes your metric with one bad frame, when it should be penalized for missing several frames and showing the user an increasingly stale image for several average-frametimes.

This isn't actually true. Looking at aveeages will in fact reflect correctly on average frame staleness over a sampling period. A single frame that's 5x too long or 4 frames that are 2x too long have the same impact on the average delta of frame staleness on the total sum.

Think of it like Little's law in queing theory , it's better to simply just look at different percentiles, like 99 or 99.9th. The average framerate correctly indicates the average staleness.

> A single frame that's 5x too long or 4 frames that are 2x too long have the same impact on the average delta of frame staleness on the total sum.

It's not clear to me: are you saying those two scenarios should be quantified as equally bad? Because it seems pretty obvious to me that the 5x outlier is qualitatively much worse.

The fact that the 5x outlier may be qualitatively worse is something that you pin down using percentiles, which is what everyone does, either 1% or 0.1% low. The argument about time-weighted frame staleness is not mathematically valid.

There’s nothing inherently mathematically special about 1% or 0.1%, you can also use 0.0042% or 0.00069% which highlights this kind of extreme outliers more.

Benchmarking sites use 1% or 0.1% because they better highlight the commodity PC hardware rather than the games bugs and architecture.

That's not true. You can't use 0.0042% because then you don't have enough frames to be statistically significant without running for a very long time. It's also true that a frame pacing issue that is so rare that it isn't even impacting the 0.1% frame time isn't going to be noticeable, really.

These benchmarks are controlled for game bugs and architecture by simply averaging over ~40-50 games, on repeatable loops.

This is a completely misguided analysis. First of all, there is nothing in the M series architecture that's going to avoid pipeline stalls, let aloe huge 70ms graphics pipeline stalls. As I've explained above, games keep multiple frame pipelines in flight at once, so you don't have these kind of issues unless there is a scheduling or I/O issue. There is no advantage here.

The only advantage is that the transfer from GPU to CPU is faster in terms of latency. This doesn't cause pipeline stalls, because as I've explained above, frames are kept in flight, so latency is not critical. At the same time, modern CPU-GPU interconnects have similar bandwidth to RAM.

Additionally, you're not the first person to have thought about frame pacing. Dozens of reviewers have full frame pacing graphs with the frame times of every frame, as well as 0.1% lows, and we simply don't see what you're describing. 100ms+ frames are not a problem on modern games and modern hardware, and when they are, it's because of some blocking read to storage or to some scheduling issue, not because of CPU/GPU speed.

The impact of link speed on 0.1% low framerates has already been investigated, and it's minimal, and there isn't even an advantage here.

> 100ms+ frames are not a problem on modern games and modern hardware, and when they are

So, you actually acknowledge that this stuff happens and 1% lows doesn’t track them properly. I can only assume you’re arguing in bad faith at this point.

There’s many causes of this stuff beyond disk IO which I can give examples for but I don’t want to get into it.

> 100ms+ frames are not a problem on modern games and modern hardware, and when they are

There’s a multitude of such bugs which ship with modern titles. Benchmarking sites don’t use the release day build of 2077 for very good reasons.

That's a problem with release day video games being unfinished buggy messes, not something that's going to change with hardware.

The only reason it wouldn't matter is that software is designed for it to not matter.

It enables software to be designed differently ie by not having to ever copy things to VRAM.

That's true to an extent, but when discussing the current performance of GPU hardware the context is necessarily software that actually exists rather than hypothetical software that could exist in the future. The high CPU-to-GPU bandwidth is not currently enabling Apple's GPUs to punch above their weight class (GFLOPS or whatever) except in scenarios where competing GPUs simply don't have enough VRAM. In a future where applications are written with Apple's architecture in mind, any advantage this provides would pretty much boil down to using more VRAM because VRAM capacity isn't as limited anymore.

Or memory slots, or dual/quad socket capability, or compatibility with the GPU card for previous gen Mac Pro. You get one multicore ARM SoC with 192GB of on-package L4 cache and integrated graphics, and that’s it.

While 192GB of RAM is more than I would need, for people looking to use 1.5TB of RAM and a pair of NVIDIA GPU they had from previous model, they’d have to go elsewhere.

Which leaves me wondering; how much engineering at Apple are happening on Mac?

Not much engineering software runs on MacOS.

The article does not say if it’s a Mac Studio or Mac Pro. I think that matters. My understanding is that there is no TDP on Apple Silicon, and that it has a lot of runway in a less constrained thermal environment like the new Mac Pro vs Mac Studio.

While I’m still surprised they didn’t put a second Ultra in the Mac Pro, I’m betting there’s a wider delta than people imagine between the two form factors.

The Mac Studio doesn't throttle so there is no extra headroom for the Mac Pro.

We’ll find out next week

Take anything from wccf with a grain of salt.

And yeah, as others point out, this is Apples to oranges. x86 desktops are great at some things, M2 Ultras are great at others, and the overlap that really matters is pretty small... Like, you have to be crazy to buy an M SoC for gaming, or buy a Nvidia GPU for workloads that won't fit in VRAM.

I tested the new game porting kit on a Mac mini M2 and was surprised I was able to run Cyberpunk 2077 and it was running really well, I can imagine that more powerful M processors will be awesome, so the argument of games might go away soon

How does it work? You install a windows game via steam and then tell it to port?

It’s basically Wine with a custom D12 translation to Metal calls. Right now it’s not really intended for end-users so the experience involves a lot of steps that aren’t consumer-friendly, but some folks are publishing wrappers that simplify the process.

I imagine that in the future, something like Steam will wrap this functionality to provide the ability to run the whole library under the toolkit. And individually-published games will do the same so they install and run with a more consumer-friendly experience.

Curious about settings for an idea of the ceiling, that game is insanely unoptimized

What is really well? Can me have a metric? What resolution and what frame rate?

Something like: M1 MBA, around 12 FPS Ultra settings or high 30s on Medium at 1440p ( max res. apparently )

So can my old 14" RTX 2060 laptop, on a 4K TV.

I have seen the demos, but I am skeptical of the actual practicality or value proposition until a 3rd party publishes some frametime benchmarks, and games out in the wild get battle tested.

Yep, but an avg. RTX 2060 laptop consumes 3x - 4x the wattage of an M2, no?

For $7,000 there should be no caveats, it should be beating every measurable metric outright.

I don't know. Nowadays a top-of-the-spec Linux workstation without any cost on top (i.e. bare-components) is around ~10k. 5995WX in the graph wccf provided alone costs ~5k.

First, a Mac Studio with M1 Ultra is $3999.

Second, people don't buy Macs only for performance. They also buy Macs for macOS, for integration between devices, for a system that is cool and quiet, for hardware acceleration of ProRes, for on-device privacy-preserving machine learning. Being a bit slower than competing AMD and Intel systems is acceptable, because you get so many other desirable properties in return.

I'd definitely consider a Mac Studio with an M2 Max or M2 Ultra, if I didn't want something portable. I would never buy a machine with a competing machine with an Intel or AMD machine, because I don't want to deal with Windows or desktop Linux.

Other people have another set of priorities and that is fine.

Maybe, but most of these comparisons are based on GPU performance (ie. games). For other workloads like machine learning, Tensor cores on NVIDIA GPUs will blow Apple Silicon GPUs out of the water. The M2 Ultra is 27 TFLOPS. The 4080't Tensor Cores are 48.7 TFLOPS in FP32, 194.9 TFLOPS in FP16, 389.9 TFLOPS in FP8 with FP16 accumulate. (IIRC on Apple Silicon GPUs FP16 performance is roughly the same as FP32.)

(There is the Neural Engine which supports lower precision, but it limited in various ways.)

Regardless, the strides that Apple has been making are impressive.

I think the exciting thing with Apple silicone and "AI" is inference, not training. Due to their unified memory, you can potentially have enormous local model inference. That's potentially much more expensive on a PC with a GPU having its own memory.

Apple have an opportunity, if they 2-4x the memory on the entry level devices (not beyond the realms of possibility), make local inference a thing available to all.

Sure, but for inference it’s much easier to work in lower precision. Training with very low precision is also possible, but is often more tricky due to numerical stability.

A lot of work is going on in 8-bit inference and even 4 bit inference. So, models that need 64 GB in FP32 can do with 16GB VRAM in FP8 or INT8, which is well within the realm of consumer NVIDIA cards. And the latest NVIDIA tensor cores will absolutely destroy Apple Silicon GPUs or the Neural Engine in 8 bit.

So, I don’t think it’s really a strong argument. And as someone who is a Mac user and a ML practitioner, I’d be very happy if they started supporting eGPUs again.

Apple Silicon has many strengths and the GPU core are fine for many ends, from games to graphics apps.

But let’s not pretend that Apple is beating NVIDIA at their own game (yet). That day might come, but currently it only leads to disappointed users in ML forums who were hyped into thinking that their vanilla M2 MacBook Airs can almost compete with a 4090 in training a deep transformer model. (Yes, that happens.)

Multiple NVIDIA GPUs also have unified memory from an application perspective, one GPU has up to 80GBs right now. They also have a proper software stack. If NVIDIA saw any credible threat there, they could ramp that up, DRAM is not beholden to Apple.

One differentiator is the amount of memory available. Apple's shared pool of memory means the GPU has potentially up to 96gb of ram at its disposal, where discrete GPU's are often limited to 8, 12, 16, etc. They mentioned in the keynote that the M2 can handle many tasks discrete GPUs cannot simply because they run out of ram.

$15,000 will buy you an A100 with 80GB of RAM at 2TB/s that will annihilate the biggest Apple chip.

Sure, but then that’s the price of multiple entire laptops. Or 2 Mac Pros. Not sure what point you’re trying to make here.

It is also the performance of 4 Mac Pros. GP was talking about tasks that dedicated GPUs supposedly could not handle. Who is is running models needing 60+ GB of VRAM on their laptops?

Some have had fun running some of the larger Llama, Vicuna & Guanaco models on their MacBooks. Mostly to experiment with local inference.

Although quantisation has now lowered the required memory, I wouldn’t be surprised if it comes in handy again in the near future.

But today that’s quite a niche use case.

FLOPS are one thing, but 192 GB of unified memory that can be used as VRAM is something else. That could be a big win on the inference side of things, where even an RTX 4090 GPU is limited to only 24 GB.

Then there's the power consumption difference to consider. This seems like one of those cases where benchmarks reveal only a fraction of the larger picture.

On purpose to keep a100 and h100 on an additional premium.

The article reports that the M2 Ultra is basically even with a 4060 Ti in OpenCL compute which they refer to as a mainstream card. It is more a notch above entry level. The "best of class" 4090 is actually 2.5x faster. But again, this is OpenCL so who knows how the two compare in a more relevant benchmark.

Regardless, wccftech is far from reliable. IIRC, /r/amd blocks links to the site.

NVIDIA still beats Apple even in efficiency, or so I have read. If you care about wattage that much, you can just underclock and undervolt your GPU. It is free and the police can not stop you.

> best of class NVIDIA discrete GPU offering

Those are nvidia's best consumer GPUs. I think the cheese grater falls into the pro segment. In that segment nvidia has the A6000s with 48GB VRAM and 91 SP TFlops compared to the 4090's 24GB and 73 SP TFlops. But that costs as much as the Mac Pro alone. And even bigger options (segmented for server/datacenter use) are available.

4090 is what it should be compared to, that's the equivalent card you'd be looking at for those workflows.