I run "npx ccusage@latest weekly" and also spot-check Claude Code sessions via "/usage".
> No consumer hardware will give you that level of latency at comparable cost.
You're right. My belief is the frontier AI companies are currently subsidizing real costs for the purpose of rapid growth customer sign ups, and that in a couple years or so local costs will become reasonable, at least for full-throttle work that's queueable.
For good latency, you would need a multi GPU server around $3-600K.
I agree the hardware will catch up and local is the future. For the average person who is sharing there privacy data out willy nilly all the time without much regard, i.e. most people, it makes no sense right now to go local vs. API / subscription given the price differential.
Local hw may get cheaper in the future, but so will dc hw, and DCs still have the advantage of 24x7 use, more efficient DC transformers, etc
And I don't need to be connected to the internet to use a local model.
I'm looking at the rate at which the models continue to improve and the pricing / rate limits in the the OpenAI/Codex/Claude/Grok subscriptions.
Local makes sense only when the rate of improvement flattens and the pricing goes way up.
I'd bet that at least the latter is going to happen, soon.
> my computer can turn on and is on 24x7.
1. Unless you are running AI queries 24x7 to max out your hardware, a piece of hardware that is idle the majority of the time (even if you had a full-time jobs that used the card 100% literally every single second of your workday, that would still be less than 1/3 utilized) is going to necessarily have way worse $ cost for you than cloud.
2. An idle PC does not use anywhere near the same power as inference running 100%, so I'm unsure why mention that your PC is on 24x7 - does that change things?
3. But even if we ignore both of the above and you had 4 or 5 40-hour per-week jobs that utilized local hardware ~100% every second, it would still be worse in terms of electricity costs, etc. than cloud due to worse AC/DC transformers for a local machine than a datacenter.
> I'm looking at the rate at which the models continue to improve and the pricing / rate limits in the the OpenAI/Codex/Claude/Grok subscriptions.
4. Last year you could have made the case that open models were cheaper, but now it's the frontier labs that are often the cheapest on the cost/task pareto curve.
> … the pricing goes way up. I'd bet that at least the latter is going to happen, soon.
5. Even if we were only comparing open cloud models vs frontier cloud models, is your argument that frontier labs will stop trying to be competitive with the open models?
6. When comparing to local, an obvious alternative would be cloud-hosted open models available from large numbers of providers already - why would their pricing go way up?
Data sovereignty always makes sense in this case because it's explicitly not about cost, but I've not seen a convincing "it's actually cheaper to do it yourself" analysis that came to that conclusion by answering all of these things up front instead of just hoping it'll somehow get better by then.
$6k is not the only cost - electricity adds up over time. And hardware can fail, redundancy is priced into the token prices and subscription plans. Maintenance is mostly negligible, but some updates/upgrades might regress. There’s a lot to SRE to keep it running at that scale.