News for hackers. Stuff that matters.
Re:Apple has new Mac app coming to make running local AI mod... by bix6 on Tuesday November 10, 2026 @07:31PM (#50448296)
Why do you need huge economies of scale though? For 90%? of queries a local model will be good enough?
Re:Apple has new Mac app coming to make running local AI mod... by Rohansi on Wednesday November 11, 2026 @01:23AM (#50450278)
For inference alone they get far better utilization of the hardware because they can keep it fully loaded with maximum concurrency. Other than that they get to collect a lot of data for training.
Re:Apple has new Mac app coming to make running local AI mod... by Foobar8568 on Wednesday November 11, 2026 @01:33AM (#50450332)
Well beside for roleplaying, local models are a bit useless, especially the ones running on end user class hardware.

Maybe when chip would infer at 10-20x the speed of a 5090 RTX, otherwise it's really..bad.

Re:Apple has new Mac app coming to make running local AI mod... by datadrivenangel on Wednesday November 11, 2026 @10:55AM (#50454887)
The larger Gemma4 and Qwen models are competattive with GPT3 / 3.5 maybe 4o, so you can run the original ChatGPT experience on a nice consumer device now.
Re:Apple has new Mac app coming to make running local AI mod... by Foobar8568 on Wednesday November 11, 2026 @11:22AM (#50455170)
I am using everyday Gemma4 31b and Qwen 3.8 27b + 3.6 35b MoE.

Beside the 35b, the others are too slow to do anything vs cloud models, unless you have a local B200+. And the Qwen 35b has the reasoning capacity of a 2yo doing maths.

I wasn't "impressed" with any models for coding until 1year ago, only then they started to be "usuable" with the fault that we know and still exists.

Re:Apple has new Mac app coming to make running local AI mod... by seunosewa on Wednesday November 11, 2026 @03:33AM (#50451001)
You'll still need a cloud AI subscription for the remaining 10%, so you might as well enjoy the superior performance. If you're cost sensitive, models like Deepseek have you covered.