You need to consider this in the context of the relevant task. Nvidia GPUs have extremely high peak performance for GEMM, but when working with LLMs, bandwidth (and RAM capacity) becomes the limiting factor. There is a reason why real ML-focused datacenter Nvidia GPUs use much wider RAM interfaces and a much higher price point. The M2 Ultra might not have the raw compute, but it has a lot of RAM and large caches.

but you also need the bandwidth to feed those units. If LLMs are your game, you need a lot of RAM and a lot of bandwidth. The 4090 can't really stretch their tensor core muscle in this scenario. That's why real ML datacenter Nvidia tech uses much wider RAM interfaces and a considerably different price point. While the M2 Ultra does not have as much