News for hackers. Stuff that matters.
Re:Apple has new Mac app coming to make running local AI mod... by gehsty on Wednesday November 11, 2026 @05:14AM (#50451688)
As models improve and hardware becomes more capable Apple are definitely well positioned.

I am super interested to see what the Apple silicon team can deliver when their full attention is turned to local LLM acceleration.

Re:Apple has new Mac app coming to make running local AI mod... by alex43578 on Wednesday November 11, 2026 @05:30AM (#50451791)
I don't see any evidence that local models (able to run on truly consumer hardware, not a DGX or multiple GPUs) will close the capability gap in a meaningful way. I have powerful hardware (multiple $K in today's prices), and only recently has Qwen Next come even close to being "useful", rather than a curiosity.

But the problem is a $20 ChatGPT/Claude subscription utterly trounces it in usability, user experience, and performance. By the time we get Opus 5.5 at home, we'll be on like Opus 7 or 8, and I'd bet Sonnet 7 or 8 will dust whatever can be run on the base MacBook of that time. It's fine if people think local 5.5 will be good enough, but it just seems like as AI keeps getting better, you'd want to use the better tooling - nobody has decided a 2010 MacBook was good enough when the M1 came out.

Re:Apple has new Mac app coming to make running local AI mod... by willis936 on Wednesday November 11, 2026 @05:37AM (#50451837)
The up and to the right trend faced issues with the end of Dennard scaling. The only way it continues is with irresponsible allocation of more natural resources to compute. I'd prefer my atoms not be turned into paperclips.
Re:Apple has new Mac app coming to make running local AI mod... by EagnaIonat on Wednesday November 11, 2026 @05:40AM (#50451850)
> will close the capability gap in a meaningful way.

I ran OpenClaw in a VM on my mac a few months back, only using local models. It was painfully slow (no MLX) but what it created was on-par with Claude at that time too.

The models are certainly powerful enough, but the frontier models are more than just an LLM model in the back.

Re:Apple has new Mac app coming to make running local AI mod... by SeanDav on Wednesday November 11, 2026 @06:07AM (#50452012)
2 things:

- As AI continues to improve, the gains will become increasingly incremental. This means that your local model will be almost identical in performance as the latest cloud version, for all but edge case problems.

- If your local model is good enough for your current needs, the only tokens you are chewing through ae utility costs and asset depreciation. Relatively cheap. It is easy to overrun even 200 /month cloud models for serious AI use.

Re:Apple has new Mac app coming to make running local AI mod... by ben_w on Wednesday November 11, 2026 @06:30AM (#50452169)
> - As AI continues to improve, the gains will become increasingly incremental. This means that your local model will be almost identical in performance as the latest cloud version, for all but edge case problems.

How much is "good enough" depends what we want it for.

Right now, the vision which was presented as a joke in (AFAICT more than one project all called) vibeOS seems like it could be viable with better local inference… but that likely needs a few more 9s of reliability even relative to current SOTA models, and we might even never get there if smaller local models always produce code which big hosted models can find vulnerabilities in.

Same if you want home inference to replace hiring a professional lawyer, architect, tutor, etc.: even though the AI has been getting passing grades in the law and medical exams since the early days, whoever gives out public liability insurance probably want to see more 9s of reliability before giving the OK.

Re:Apple has new Mac app coming to make running local AI mod... by radlad on Wednesday November 11, 2026 @09:43AM (#50454059)
> It is easy to overrun even 200 /month cloud models for serious AI use.

Not with the tok/s I get.

Re:Apple has new Mac app coming to make running local AI mod... by cribbles on Wednesday November 11, 2026 @06:15AM (#50452062)
I agree with you, fwiw, but Apple's own consumer hardcore is not the choice of metaphor I would go with here - the M1 MacBook Pro and the group of iPhones that came out around the same time were the exact moment when Apple devices hit a "good enough" asymptote for many people (myself included)
Re:Apple has new Mac app coming to make running local AI mod... by xdertz on Wednesday November 11, 2026 @08:04AM (#50452931)
If you are comparing pure quality sure but don't forget quantity. It is very easy to blow through a weekly quota in one day. Fully local means you have unlimited usage which is something no provider gives you on a subscription.

When comparing API prices and privacy concerns, local might win.

Re:Apple has new Mac app coming to make running local AI mod... by bearjaws on Wednesday November 11, 2026 @08:11AM (#50452999)
> Fully local means you have unlimited usage

20tk/s is not unlimited when you have a lifespan that isn't infinite.

Re:Apple has new Mac app coming to make running local AI mod... by mycall on Wednesday November 11, 2026 @08:56AM (#50453484)
> Claude subscription

Not good for PII/PPI

Re:Apple has new Mac app coming to make running local AI mod... by gehsty on Wednesday November 11, 2026 @09:05AM (#50453591)
The tension over local / remote is a tale as old as time in computing, when the technology settles down both will continue to exist with different use cases…

in 5yrs time running an M12 with a Bionic LLM Engine (or whatever they call it) may be good enough for a very very good AI experience. Enterprise / professionals will likely still pay for frontier models.

Re:Apple has new Mac app coming to make running local AI mod... by Rover222 on Wednesday November 11, 2026 @11:30AM (#50455290)
Yeah I think it’s peak nerd bubble to think local models are going to wipe out the value prop of Anthropic.
Re:Apple has new Mac app coming to make running local AI mod... by mrtksn on Wednesday November 11, 2026 @05:41AM (#50451857)
Sure, Apple has an opportunity here but I have no confidence that they won't screw it up.

For starters, they need to let me get rid of their AI stuff and free up the space. Apple aggressively pushing their AI down the throat of their users means that they suck at it but trying to make it happen using their muscles in other areas.

Just like people figured out how to make their Android TV not suck, it suggests that some time it the future people can make their OS not suck and this will make macOS irrelevant if Apple doesn't fix it first. Unfortunately for Apple, they operate in a system where they must keep growing their numbers.

Such a great company, its a shame that they couldn't figure out this AI stuff. Fingers crossed that this tool actually fixes that by just giving ap and let people do whatever they want to do as efficiently as they can on the brilliant hardware Apple creates.

Re:Apple has new Mac app coming to make running local AI mod... by ethbr1 on Wednesday November 11, 2026 @06:53AM (#50452333)
I'd add to this that Apple's ability to support a third-party ecosystem with stable development interfaces to their hardware has been at best "mixed" over the years.

Especially in the modern era, the firmware/software side of macOS definitely doesn't seem like a priority.

Re:Apple has new Mac app coming to make running local AI mod... by mark_l_watson on Wednesday November 11, 2026 @09:48AM (#50454127)
Their hardware is especially good for MOE models, less suitable for dense. I use Laguna XS 2.1 6bit and Qwen-3.8-next-flash 2.6bit most because they are MOEs and run efficiently. I love Qwen-3.7:27B but as a dense model I only use it in “run something and walk away” use cases: slow.