A 256GB Mac Pro can dedicate almost all of that to AI.
Their GPUs are the weak spot. The Nvidia 4080 is 2.5x faster than the M2, and the A100 is 15 times faster.
Not a single benchmark in the world has supported Apple's claim that the GPU in the M2 is that powerful. It's just yet another cute embedded GPU that does the job, but nothing more. It's made to push out 8K frames really fast, which it does because of UMA, but want demanding task will have it be eaten alive by any real GPU.
But in any event, yes, that was my point. UMA is a huge advantage, the GPU itself is too weak to be serious.
But it’s a lot easier to drop a dramatically beefier GPU into a new design than it is to update the entire platform for UMA. Apple has a huge opportunity here… whether tbey pursue it or not remains to be seen.
Pursue what though?
UMA is cool, but kinda meaningless if the majority of Macbooks are min-spec. That leaves you with 4-5gb of VRAM, assuming you've left nothing open. What is Apple going to do with that UMA that other manufacturers cannot?
It's certainly nice that 128gb Macs exist for models that might be too big to otherwise load into memory. It's useless for production inferencing though, and I struggle to imagine the "opportunities" they're missing out on here.
A Mac variant that trades CPU cores for GPU/ML cores while having 192GB+ of UMA memory.
> I struggle to imagine the “opportunities”
Two of them: 1) academic / R&D compute, where people could have at least A6000 class GPU on the desktop, and 2) cloud inference servers, probably for Apple’s own services.
I’m not saying they will or should do those things, just that the apple silicon arch is well positioned if they choose to. Bolting on exponentially better GPU is not especially difficult, and they’ve got an OS that would bring existing apps and libraries right over.
Look at it this way: is there a path to UMA on Windows / Linux? If not, those systems will always duplicate RAM and require users to decide in advance whether to allocate RAM budget to OS or ML.
Whichever way you look at it though, neither of those are really opportunities. Apple boxed themselves into the consumer market, and now has to compete with professionally-priced products.