The framework desktop is quite cool, but those Ryzen Max CPUs are still a pretty poor competitor to Apple's chips if what you care about it running an LLM. Ryzen Max tops out at 256 GB/s of memory bandwidth, whereas an M4 Max can hit 560 GB/s of bandwidth.

So even if the model fits in the memory buffer on the Ryzen Max, you're still going to hit something like half the tokens/second just because the GPU will be sitting around waiting for data.

Personally, I'd rather have the Framework machine, but if running local LLMs is your main goal, the offerings from Apple are very compelling, even when you adjust for the higher price on the Apple machine.

There's also the DGX Spark. Granted, its price has been going up recently alongside everything else that has memory in it.

I haven't heard a single good think about DGX Spark from anyone using it, so I'd be pretty wary about that.

That also has pretty poor memory bandwidth. 283GB/s I think.

Yeah. The main selling point I'd say is the onboard ConnectX-7 hardware.

128gb is the max RAM that the current Strix Halo supports with ~250GB/s of bandwidth. The Mac Studio is 256GB max and ~900GB/s of memory bandwidth. They are in different categories of performance, even price-per-dollar is worse. (~$2700 for Framework Desktop vs $7500 for Mac Studio M3 Ultra)

128GB*

Thanks for spotting the mistake. No Idea how I got to 192

For what it's worth, I really wish that was the actual number.