yes, but you pay the Ultra plan, that's the whole point, isn't it? Let's be honest also - qwen3.8:27b-mlx is nowhere near Sonnet, both in size and cap, and if you do $10k a week of usage, you should know that.
I've tried it out on a 512GB dram server with 2*3090s and it works really quite well. Draws enough power to cost something similar to Luna, though.
I don't think any of these self hosted setups are a good deal when you look at the NPV of the capex plus the power costs, but if you already own the hardware it can be okay or even a slight cost savings.
Claude currently does these via API pricing, not Claude Ultra personal.
$6k is not the only cost - electricity adds up over time. And hardware can fail, redundancy is priced into the token prices and subscription plans. Maintenance is mostly negligible, but some updates/upgrades might regress. There’s a lot to SRE to keep it running at that scale.
I run "npx ccusage@latest weekly" and also spot-check Claude Code sessions via "/usage".
> No consumer hardware will give you that level of latency at comparable cost.
You're right. My belief is the frontier AI companies are currently subsidizing real costs for the purpose of rapid growth customer sign ups, and that in a couple years or so local costs will become reasonable, at least for full-throttle work that's queueable.
For good latency, you would need a multi GPU server around $3-600K.
I agree the hardware will catch up and local is the future. For the average person who is sharing there privacy data out willy nilly all the time without much regard, i.e. most people, it makes no sense right now to go local vs. API / subscription given the price differential.
Local hw may get cheaper in the future, but so will dc hw, and DCs still have the advantage of 24x7 use, more efficient DC transformers, etc
And I don't need to be connected to the internet to use a local model.
I'm looking at the rate at which the models continue to improve and the pricing / rate limits in the the OpenAI/Codex/Claude/Grok subscriptions.
Local makes sense only when the rate of improvement flattens and the pricing goes way up.
I'd bet that at least the latter is going to happen, soon.
> my computer can turn on and is on 24x7.
1. Unless you are running AI queries 24x7 to max out your hardware, a piece of hardware that is idle the majority of the time (even if you had a full-time jobs that used the card 100% literally every single second of your workday, that would still be less than 1/3 utilized) is going to necessarily have way worse $ cost for you than cloud.
2. An idle PC does not use anywhere near the same power as inference running 100%, so I'm unsure why mention that your PC is on 24x7 - does that change things?
3. But even if we ignore both of the above and you had 4 or 5 40-hour per-week jobs that utilized local hardware ~100% every second, it would still be worse in terms of electricity costs, etc. than cloud due to worse AC/DC transformers for a local machine than a datacenter.
> I'm looking at the rate at which the models continue to improve and the pricing / rate limits in the the OpenAI/Codex/Claude/Grok subscriptions.
4. Last year you could have made the case that open models were cheaper, but now it's the frontier labs that are often the cheapest on the cost/task pareto curve.
> … the pricing goes way up. I'd bet that at least the latter is going to happen, soon.
5. Even if we were only comparing open cloud models vs frontier cloud models, is your argument that frontier labs will stop trying to be competitive with the open models?
6. When comparing to local, an obvious alternative would be cloud-hosted open models available from large numbers of providers already - why would their pricing go way up?
Data sovereignty always makes sense in this case because it's explicitly not about cost, but I've not seen a convincing "it's actually cheaper to do it yourself" analysis that came to that conclusion by answering all of these things up front instead of just hoping it'll somehow get better by then.
A Mac Studio high end is $30k one-time versus my current run rate of $10k/week for Claude Code (if I paid API rates rather than Ultra plan). he comparison is a general ballpark not an exact equivalent.
I currently run 128GB with qwen3.8:27b-mlx. It hits the heat limit of the Mac when working on a medium-size codebase for test-driven compliance such as security auditing, that need to deeply evaluate the whole codebase.