- 4090: Going for $7500
- 64GB M5 Max: Starts at $3,800
- R9700: $2700
But it's also an asset that barely depreciates!
Especially right now — you could buy and extensively use an M5 Max for a year and sell it for almost exactly the same amount at the end. Maybe even more, at the current rate.
Furthermore, it can do other things for you: host your VMs, edit high def video, whatever. Even if it stops being useful to serve LLMs, it might be useful to run as a server for other tasks for a whole decade.
Depends what you want, doesn't it? Or are we all expected to conform?
For me personally, right now I have an M1 Max macbook pro, and that is my local AI machine, because my needs are exploratory and I want to run it here. I want to learn by comparing models. I want to learn how to run them and advise others how to (local inference is not a zero-sized market).
My next step up will be to take the apps I'm looking at building that run against these small models, and host those apps online with some sort of cloud inference — openrouter, digitalocean, cloudflare. Again — if I can avoid it, my apps won't develop a dependency on any AI provider.
If I ever find I want to vaguely describe some desktop application before bed and come down the next morning to an aggravating slop version of it, I'll consider Claude.
As for a local machine, I've looked at the M5 Max and the R9700; haven't pushed the button yet because I don't need it enough.
I've also looked at the M5 Pro, which in practice is about as fast at token generation as my M1 Max, and smokes it at prefill. That alone might be enough for a "leave it running" mini AI box, because agentic coding is so prefill-heavy. But I am a laptop person for the next six months or so I think.
As an aside there's that other thing which is somewhat interesting, though constrained:
Uses the 5090 mobile chip, which is pretty good speedwise (about the same as the R9700) but 24GB. Small, tidy, on-prem. I think they have missed the boat on pricing, but if they can ship in volume, maybe not.
Having a 4090 at home running agents all day is not a $20pcm level of usage, so take the $100 plan. Looking at the S-1 filing, today they are not even close to breakeven. It is reasonable to assume that will become a $300-400 pcm plan.
The launch MSRP of a 4090 was $1600. There has been significant inflation since then, so say $2000. Eventually supply will meet demand and you'll be able to buy one for that price again, or lower due to the economies of scale they must have now.
So, in 2-3 years time once it has all settled, you pay off your 4090 by saving 5-6 months of Claude usage. Maybe the local model isn't good enough for complete replacement, but all it needs to do is drop you to the $20 plan to be worth it. I think this is a large part of why investors are getting jittery now.
You can surely build an LLM server with an R9700 and 32GB of mainboard RAM for less than the 64GB M5 Max Apple Studio, which is probably the next most competitive model up. It'll be better than a Strix Halo but less flexible.