Re:Apple has new Mac app coming to make running local AI mod...
by bix6 on Tuesday November 10, 2026 @07:31PM
(#50448296)
Why do you need huge economies of scale though? For 90%? of queries a local model will be good enough?
[ Parent
| on HN ]
Re:Apple has new Mac app coming to make running local AI mod...
by Rohansi on Wednesday November 11, 2026 @01:23AM
(#50450278)
For inference alone they get far better utilization of the hardware because they can keep it fully loaded with maximum concurrency. Other than that they get to collect a lot of data for training.
Re:Apple has new Mac app coming to make running local AI mod...
by Foobar8568 on Wednesday November 11, 2026 @01:33AM
(#50450332)
Well beside for roleplaying, local models are a bit useless, especially the ones running on end user class hardware.
Maybe when chip would infer at 10-20x the speed of a 5090 RTX, otherwise it's really..bad.
Re:Apple has new Mac app coming to make running local AI mod...
by datadrivenangel on Wednesday November 11, 2026 @10:55AM
(#50454887)
The larger Gemma4 and Qwen models are competattive with GPT3 / 3.5 maybe 4o, so you can run the original ChatGPT experience on a nice consumer device now.
Re:Apple has new Mac app coming to make running local AI mod...
by Foobar8568 on Wednesday November 11, 2026 @11:22AM
(#50455170)
I am using everyday Gemma4 31b and Qwen 3.8 27b + 3.6 35b MoE.
Beside the 35b, the others are too slow to do anything vs cloud models, unless you have a local B200+. And the Qwen 35b has the reasoning capacity of a 2yo doing maths.
I wasn't "impressed" with any models for coding until 1year ago, only then they started to be "usuable" with the fault that we know and still exists.
Re:Apple has new Mac app coming to make running local AI mod...
by seunosewa on Wednesday November 11, 2026 @03:33AM
(#50451001)
You'll still need a cloud AI subscription for the remaining 10%, so you might as well enjoy the superior performance. If you're cost sensitive, models like Deepseek have you covered.