News for hackers. Stuff that matters.
Re:Apple has new Mac app coming to make running local AI mod... by jph on Wednesday November 11, 2026 @03:48AM (#50451085)
> How did you arrive at that number?

I run "npx ccusage@latest weekly" and also spot-check Claude Code sessions via "/usage".

> No consumer hardware will give you that level of latency at comparable cost.

You're right. My belief is the frontier AI companies are currently subsidizing real costs for the purpose of rapid growth customer sign ups, and that in a couple years or so local costs will become reasonable, at least for full-throttle work that's queueable.

Re:Apple has new Mac app coming to make running local AI mod... by adi_kurian on Wednesday November 11, 2026 @02:51PM (#50457739)
I did the math once, to run Kimi K3 on a Mac, you'd need to daisy chain 5 Mac Studio M3 Ultras. So $125K. And the latency would be bad.

For good latency, you would need a multi GPU server around $3-600K.

I agree the hardware will catch up and local is the future. For the average person who is sharing there privacy data out willy nilly all the time without much regard, i.e. most people, it makes no sense right now to go local vs. API / subscription given the price differential.

Re:Apple has new Mac app coming to make running local AI mod... by no-name-here on Sunday November 15, 2026 @03:09AM (#50505030)
> no sense right now

Local hw may get cheaper in the future, but so will dc hw, and DCs still have the advantage of 24x7 use, more efficient DC transformers, etc

Re:Apple has new Mac app coming to make running local AI mod... by adi_kurian on Monday November 16, 2026 @06:35PM (#50525167)
Unless the grid goes down, my computer can turn on and is on 24x7.

And I don't need to be connected to the internet to use a local model.

I'm looking at the rate at which the models continue to improve and the pricing / rate limits in the the OpenAI/Codex/Claude/Grok subscriptions.

Local makes sense only when the rate of improvement flattens and the pricing goes way up.

I'd bet that at least the latter is going to happen, soon.

Re:Apple has new Mac app coming to make running local AI mod... by no-name-here on Tuesday November 17, 2026 @01:42AM (#50528200)
>> DCs still have the advantage of 24x7 use, more efficient DC transformers

> my computer can turn on and is on 24x7.

1. Unless you are running AI queries 24x7 to max out your hardware, a piece of hardware that is idle the majority of the time (even if you had a full-time jobs that used the card 100% literally every single second of your workday, that would still be less than 1/3 utilized) is going to necessarily have way worse $ cost for you than cloud.

2. An idle PC does not use anywhere near the same power as inference running 100%, so I'm unsure why mention that your PC is on 24x7 - does that change things?

3. But even if we ignore both of the above and you had 4 or 5 40-hour per-week jobs that utilized local hardware ~100% every second, it would still be worse in terms of electricity costs, etc. than cloud due to worse AC/DC transformers for a local machine than a datacenter.

> I'm looking at the rate at which the models continue to improve and the pricing / rate limits in the the OpenAI/Codex/Claude/Grok subscriptions.

4. Last year you could have made the case that open models were cheaper, but now it's the frontier labs that are often the cheapest on the cost/task pareto curve.

> … the pricing goes way up. I'd bet that at least the latter is going to happen, soon.

5. Even if we were only comparing open cloud models vs frontier cloud models, is your argument that frontier labs will stop trying to be competitive with the open models?

6. When comparing to local, an obvious alternative would be cloud-hosted open models available from large numbers of providers already - why would their pricing go way up?

Re:Apple has new Mac app coming to make running local AI mod... by zamadatix on Wednesday November 11, 2026 @08:53PM (#50462016)
Even if one believes the price of models will only go up from frontier AI companies AND believe the local AI frontier will be fast enough at a good enough profile, it still doesn't explain how you're going to beat the price of a place hosting open weight models at scale on shared hardware always maxed out 100% in a place with cheap electricity by buying a single platform up front for 3 years.

Data sovereignty always makes sense in this case because it's explicitly not about cost, but I've not seen a convincing "it's actually cheaper to do it yourself" analysis that came to that conclusion by answering all of these things up front instead of just hoping it'll somehow get better by then.