News for hackers. Stuff that matters.
Re:Apple has new Mac app coming to make running local AI mod... by jph on Wednesday November 11, 2026 @02:10AM (#50450531)
If you ask who would pay for Apple local models, I would.

A Mac Studio high end is $30k one-time versus my current run rate of $10k/week for Claude Code (if I paid API rates rather than Ultra plan). he comparison is a general ballpark not an exact equivalent.

I currently run 128GB with qwen3.8:27b-mlx. It hits the heat limit of the Mac when working on a medium-size codebase for test-driven compliance such as security auditing, that need to deeply evaluate the whole codebase.

Re:Apple has new Mac app coming to make running local AI mod... by larodi on Wednesday November 11, 2026 @02:20AM (#50450592)
> if I paid API rates rather than Ultra plan

yes, but you pay the Ultra plan, that's the whole point, isn't it? Let's be honest also - qwen3.8:27b-mlx is nowhere near Sonnet, both in size and cap, and if you do $10k a week of usage, you should know that.

Re:Apple has new Mac app coming to make running local AI mod... by dgacmu on Wednesday November 11, 2026 @10:28AM (#50454565)
The 512GB server he specified would be running something more like qwen3.8-flash-next, which is probably competitive with sonnet 5 (but still loses to 5.5).

I've tried it out on a 512GB dram server with 2*3090s and it works really quite well. Draws enough power to cost something similar to Luna, though.

Re:Apple has new Mac app coming to make running local AI mod... by larodi on Wednesday November 11, 2026 @04:25PM (#50458903)
Still u can never beat the cost of electricity unless saturated with batches inference, speculative decode, maybe beam search and what not.
Re:Apple has new Mac app coming to make running local AI mod... by dgacmu on Wednesday November 11, 2026 @08:39PM (#50461897)
Not really. My setup is just a hair cheaper than paying API prices for Luna (and it will do security audits for me). The mac would be even better on power.

I don't think any of these self hosted setups are a good deal when you look at the NPV of the capex plus the power costs, but if you already own the hardware it can be okay or even a slight cost savings.

Re:Apple has new Mac app coming to make running local AI mod... by zaphirplane on Wednesday November 11, 2026 @02:22AM (#50450612)
[delayed]
Re:Apple has new Mac app coming to make running local AI mod... by jph on Wednesday November 11, 2026 @03:55AM (#50451118)
Break even is likely for API pricing. For example one of the projects is a web app that's processing user input, making AI suggestions, generating reports, doing ongoing observability, and running batch processing overnight.

Claude currently does these via API pricing, not Claude Ultra personal.

Re:Apple has new Mac app coming to make running local AI mod... by jmaker on Wednesday November 11, 2026 @03:10AM (#50450883)
$10k for Sonnet per week? How did you arrive at that number? It’s quite a huge inference volume. No consumer hardware will give you that level of latency at comparable cost.

$6k is not the only cost - electricity adds up over time. And hardware can fail, redundancy is priced into the token prices and subscription plans. Maintenance is mostly negligible, but some updates/upgrades might regress. There’s a lot to SRE to keep it running at that scale.

Re:Apple has new Mac app coming to make running local AI mod... by jph on Wednesday November 11, 2026 @03:48AM (#50451085)
> How did you arrive at that number?

I run "npx ccusage@latest weekly" and also spot-check Claude Code sessions via "/usage".

> No consumer hardware will give you that level of latency at comparable cost.

You're right. My belief is the frontier AI companies are currently subsidizing real costs for the purpose of rapid growth customer sign ups, and that in a couple years or so local costs will become reasonable, at least for full-throttle work that's queueable.

Re:Apple has new Mac app coming to make running local AI mod... by adi_kurian on Wednesday November 11, 2026 @02:51PM (#50457739)
I did the math once, to run Kimi K3 on a Mac, you'd need to daisy chain 5 Mac Studio M3 Ultras. So $125K. And the latency would be bad.

For good latency, you would need a multi GPU server around $3-600K.

I agree the hardware will catch up and local is the future. For the average person who is sharing there privacy data out willy nilly all the time without much regard, i.e. most people, it makes no sense right now to go local vs. API / subscription given the price differential.

Re:Apple has new Mac app coming to make running local AI mod... by no-name-here on Sunday November 15, 2026 @03:09AM (#50505030)
> no sense right now

Local hw may get cheaper in the future, but so will dc hw, and DCs still have the advantage of 24x7 use, more efficient DC transformers, etc

Re:Apple has new Mac app coming to make running local AI mod... by adi_kurian on Monday November 16, 2026 @06:35PM (#50525167)
Unless the grid goes down, my computer can turn on and is on 24x7.

And I don't need to be connected to the internet to use a local model.

I'm looking at the rate at which the models continue to improve and the pricing / rate limits in the the OpenAI/Codex/Claude/Grok subscriptions.

Local makes sense only when the rate of improvement flattens and the pricing goes way up.

I'd bet that at least the latter is going to happen, soon.

Re:Apple has new Mac app coming to make running local AI mod... by no-name-here on Tuesday November 17, 2026 @01:42AM (#50528200)
>> DCs still have the advantage of 24x7 use, more efficient DC transformers

> my computer can turn on and is on 24x7.

1. Unless you are running AI queries 24x7 to max out your hardware, a piece of hardware that is idle the majority of the time (even if you had a full-time jobs that used the card 100% literally every single second of your workday, that would still be less than 1/3 utilized) is going to necessarily have way worse $ cost for you than cloud.

2. An idle PC does not use anywhere near the same power as inference running 100%, so I'm unsure why mention that your PC is on 24x7 - does that change things?

3. But even if we ignore both of the above and you had 4 or 5 40-hour per-week jobs that utilized local hardware ~100% every second, it would still be worse in terms of electricity costs, etc. than cloud due to worse AC/DC transformers for a local machine than a datacenter.

> I'm looking at the rate at which the models continue to improve and the pricing / rate limits in the the OpenAI/Codex/Claude/Grok subscriptions.

4. Last year you could have made the case that open models were cheaper, but now it's the frontier labs that are often the cheapest on the cost/task pareto curve.

> … the pricing goes way up. I'd bet that at least the latter is going to happen, soon.

5. Even if we were only comparing open cloud models vs frontier cloud models, is your argument that frontier labs will stop trying to be competitive with the open models?

6. When comparing to local, an obvious alternative would be cloud-hosted open models available from large numbers of providers already - why would their pricing go way up?

Re:Apple has new Mac app coming to make running local AI mod... by zamadatix on Wednesday November 11, 2026 @08:53PM (#50462016)
Even if one believes the price of models will only go up from frontier AI companies AND believe the local AI frontier will be fast enough at a good enough profile, it still doesn't explain how you're going to beat the price of a place hosting open weight models at scale on shared hardware always maxed out 100% in a place with cheap electricity by buying a single platform up front for 3 years.

Data sovereignty always makes sense in this case because it's explicitly not about cost, but I've not seen a convincing "it's actually cheaper to do it yourself" analysis that came to that conclusion by answering all of these things up front instead of just hoping it'll somehow get better by then.

Re:Apple has new Mac app coming to make running local AI mod... by jann on Wednesday November 11, 2026 @10:15AM (#50454423)
To be fair, having multiple Claude subscriptions is also against the ToS afaik, so you’d have a much higher run rate.
Re:Apple has new Mac app coming to make running local AI mod... by jph on Wednesday November 11, 2026 @04:31PM (#50458984)
Anthropic tells me I can have two Ultra subscriptions. FWIW I'm doing open source.
Re:Apple has new Mac app coming to make running local AI mod... by larodi on Friday November 13, 2026 @06:04AM (#50481779)
I can easily see everyone on my team having a Claude subscription, and then we can orchestrate and federate to spend the unused limits. I don’t see how this breaks toc, given we don’t resell. And 1000€ (or dollars) in Anthropic subscription buys billions if not trillions of tokens, if used right.
Re:Apple has new Mac app coming to make running local AI mod... by larodi on Friday November 13, 2026 @06:02AM (#50481753)
Why is it that you don’t include costs for the engineer catering to 512gigs system, potentially well saturated in usage? Anyone who understands inference at this level is at least 5k a month.