In any case how are you going to fit 50+GB in two (24+24 GB) Nvidia cards without swapping to disk?
> In any case how are you going to fit 50+GB in two (theoretically 24+24 GB) Nvidia cards
No one was talking about only two cards.
That post read like a joke comparison and still does. Can you elaborate how it is relevant?
The parent of my initial comment in this thread said: "For inference, Apple chips are great due to a high memory bandwidth... It's a cost effective option if you need a lot of memory plus a high bandwidth."
My post was attempting to explain at a high level how 1) Apple SoCs do not really have high memory bandwidth compared to a cluster of GPUs, and 2) you can actually build that cluster of GPUs for the same cost or cheaper than a loaded Mac Studio, and it will drastically outperform the Mac.
If you want specifics on how to build such a GPU cluster, you can search for "ROMED8-2T 3090" for some examples.
I hope this helps.
Apple's chips have the advantage of being able to be specced out with tons of RAM, but performance isn't going to be in the same ballpark of even fairly old Nvidia chips.