News for hackers. Stuff that matters.
Re:Apple has new Mac app coming to make running local AI mod... by larodi on Wednesday November 11, 2026 @02:20AM (#50450592)
> if I paid API rates rather than Ultra plan

yes, but you pay the Ultra plan, that's the whole point, isn't it? Let's be honest also - qwen3.8:27b-mlx is nowhere near Sonnet, both in size and cap, and if you do $10k a week of usage, you should know that.

Re:Apple has new Mac app coming to make running local AI mod... by dgacmu on Wednesday November 11, 2026 @10:28AM (#50454565)
The 512GB server he specified would be running something more like qwen3.8-flash-next, which is probably competitive with sonnet 5 (but still loses to 5.5).

I've tried it out on a 512GB dram server with 2*3090s and it works really quite well. Draws enough power to cost something similar to Luna, though.

Re:Apple has new Mac app coming to make running local AI mod... by larodi on Wednesday November 11, 2026 @04:25PM (#50458903)
Still u can never beat the cost of electricity unless saturated with batches inference, speculative decode, maybe beam search and what not.
Re:Apple has new Mac app coming to make running local AI mod... by dgacmu on Wednesday November 11, 2026 @08:39PM (#50461897)
Not really. My setup is just a hair cheaper than paying API prices for Luna (and it will do security audits for me). The mac would be even better on power.

I don't think any of these self hosted setups are a good deal when you look at the NPV of the capex plus the power costs, but if you already own the hardware it can be okay or even a slight cost savings.