Re:Apple has new Mac app coming to make running local AI mod...
by larodi on Wednesday November 11, 2026 @04:25PM
(#50458903)
Still u can never beat the cost of electricity unless saturated with batches inference, speculative decode, maybe beam search and what not.
Re:Apple has new Mac app coming to make running local AI mod...
by dgacmu on Wednesday November 11, 2026 @08:39PM
(#50461897)
Not really. My setup is just a hair cheaper than paying API prices for Luna (and it will do security audits for me). The mac would be even better on power.
I don't think any of these self hosted setups are a good deal when you look at the NPV of the capex plus the power costs, but if you already own the hardware it can be okay or even a slight cost savings.
I've tried it out on a 512GB dram server with 2*3090s and it works really quite well. Draws enough power to cost something similar to Luna, though.