News for hackers. Stuff that matters.
Re:Apple has new Mac app coming to make running local AI mod... by alex43578 on Wednesday November 11, 2026 @05:30AM (#50451791)
I don't see any evidence that local models (able to run on truly consumer hardware, not a DGX or multiple GPUs) will close the capability gap in a meaningful way. I have powerful hardware (multiple $K in today's prices), and only recently has Qwen Next come even close to being "useful", rather than a curiosity.

But the problem is a $20 ChatGPT/Claude subscription utterly trounces it in usability, user experience, and performance. By the time we get Opus 5.5 at home, we'll be on like Opus 7 or 8, and I'd bet Sonnet 7 or 8 will dust whatever can be run on the base MacBook of that time. It's fine if people think local 5.5 will be good enough, but it just seems like as AI keeps getting better, you'd want to use the better tooling - nobody has decided a 2010 MacBook was good enough when the M1 came out.

Re:Apple has new Mac app coming to make running local AI mod... by willis936 on Wednesday November 11, 2026 @05:37AM (#50451837)
The up and to the right trend faced issues with the end of Dennard scaling. The only way it continues is with irresponsible allocation of more natural resources to compute. I'd prefer my atoms not be turned into paperclips.
Re:Apple has new Mac app coming to make running local AI mod... by EagnaIonat on Wednesday November 11, 2026 @05:40AM (#50451850)
> will close the capability gap in a meaningful way.

I ran OpenClaw in a VM on my mac a few months back, only using local models. It was painfully slow (no MLX) but what it created was on-par with Claude at that time too.

The models are certainly powerful enough, but the frontier models are more than just an LLM model in the back.

Re:Apple has new Mac app coming to make running local AI mod... by SeanDav on Wednesday November 11, 2026 @06:07AM (#50452012)
2 things:

- As AI continues to improve, the gains will become increasingly incremental. This means that your local model will be almost identical in performance as the latest cloud version, for all but edge case problems.

- If your local model is good enough for your current needs, the only tokens you are chewing through ae utility costs and asset depreciation. Relatively cheap. It is easy to overrun even 200 /month cloud models for serious AI use.

Re:Apple has new Mac app coming to make running local AI mod... by ben_w on Wednesday November 11, 2026 @06:30AM (#50452169)
> - As AI continues to improve, the gains will become increasingly incremental. This means that your local model will be almost identical in performance as the latest cloud version, for all but edge case problems.

How much is "good enough" depends what we want it for.

Right now, the vision which was presented as a joke in (AFAICT more than one project all called) vibeOS seems like it could be viable with better local inference… but that likely needs a few more 9s of reliability even relative to current SOTA models, and we might even never get there if smaller local models always produce code which big hosted models can find vulnerabilities in.

Same if you want home inference to replace hiring a professional lawyer, architect, tutor, etc.: even though the AI has been getting passing grades in the law and medical exams since the early days, whoever gives out public liability insurance probably want to see more 9s of reliability before giving the OK.

Re:Apple has new Mac app coming to make running local AI mod... by radlad on Wednesday November 11, 2026 @09:43AM (#50454059)
> It is easy to overrun even 200 /month cloud models for serious AI use.

Not with the tok/s I get.

Re:Apple has new Mac app coming to make running local AI mod... by cribbles on Wednesday November 11, 2026 @06:15AM (#50452062)
I agree with you, fwiw, but Apple's own consumer hardcore is not the choice of metaphor I would go with here - the M1 MacBook Pro and the group of iPhones that came out around the same time were the exact moment when Apple devices hit a "good enough" asymptote for many people (myself included)
Re:Apple has new Mac app coming to make running local AI mod... by xdertz on Wednesday November 11, 2026 @08:04AM (#50452931)
If you are comparing pure quality sure but don't forget quantity. It is very easy to blow through a weekly quota in one day. Fully local means you have unlimited usage which is something no provider gives you on a subscription.

When comparing API prices and privacy concerns, local might win.

Re:Apple has new Mac app coming to make running local AI mod... by bearjaws on Wednesday November 11, 2026 @08:11AM (#50452999)
> Fully local means you have unlimited usage

20tk/s is not unlimited when you have a lifespan that isn't infinite.

Re:Apple has new Mac app coming to make running local AI mod... by mycall on Wednesday November 11, 2026 @08:56AM (#50453484)
> Claude subscription

Not good for PII/PPI

Re:Apple has new Mac app coming to make running local AI mod... by gehsty on Wednesday November 11, 2026 @09:05AM (#50453591)
The tension over local / remote is a tale as old as time in computing, when the technology settles down both will continue to exist with different use cases…

in 5yrs time running an M12 with a Bionic LLM Engine (or whatever they call it) may be good enough for a very very good AI experience. Enterprise / professionals will likely still pay for frontier models.

Re:Apple has new Mac app coming to make running local AI mod... by Rover222 on Wednesday November 11, 2026 @11:30AM (#50455290)
Yeah I think it’s peak nerd bubble to think local models are going to wipe out the value prop of Anthropic.