But it's also an asset that barely depreciates!

Especially right now — you could buy and extensively use an M5 Max for a year and sell it for almost exactly the same amount at the end. Maybe even more, at the current rate.

Furthermore, it can do other things for you: host your VMs, edit high def video, whatever. Even if it stops being useful to serve LLMs, it might be useful to run as a server for other tasks for a whole decade.

Depends what you want, doesn't it? Or are we all expected to conform?

For me personally, right now I have an M1 Max macbook pro, and that is my local AI machine, because my needs are exploratory and I want to run it here. I want to learn by comparing models. I want to learn how to run them and advise others how to (local inference is not a zero-sized market).

My next step up will be to take the apps I'm looking at building that run against these small models, and host those apps online with some sort of cloud inference — openrouter, digitalocean, cloudflare. Again — if I can avoid it, my apps won't develop a dependency on any AI provider.

If I ever find I want to vaguely describe some desktop application before bed and come down the next morning to an aggravating slop version of it, I'll consider Claude.

As for a local machine, I've looked at the M5 Max and the R9700; haven't pushed the button yet because I don't need it enough.

I've also looked at the M5 Pro, which in practice is about as fast at token generation as my M1 Max, and smokes it at prefill. That alone might be enough for a "leave it running" mini AI box, because agentic coding is so prefill-heavy. But I am a laptop person for the next six months or so I think.

As an aside there's that other thing which is somewhat interesting, though constrained:

https://www.olares.com/one

Uses the 5090 mobile chip, which is pretty good speedwise (about the same as the R9700) but 24GB. Small, tidy, on-prem. I think they have missed the boat on pricing, but if they can ship in volume, maybe not.