News for hackers. Stuff that matters.

tylermhall

Re:Local AI is real now. And it's blowing my mind on Wednesday October 07, 2026 @03:28PM (#49997657)
I've not really used it to replace any frontier. More that DeepSeek is something that gets turned on to do useful work when I need it. The smaller models I leave on 24/7 for work that needs very little intelligence, constantly. I run IoT development, so it's clef-flash first, then gemma or qwen 35 A3B to do SOME sort of thought on "does this look weird," relate it to the firmware, and build a report on the fleet. The small ones are fast and cheap on memory; Flash Next takes 99 GB and is half the speed, which is the whole reason for the slots on one box.

My purpose for the stack is not to eliminate frontier. I don't think that's really useful goal to have. My hope is to use it as a "new" place in my stack that I'd never want to pay frontier pricing for. It's private, always on, powered by solar, and I can throw a sort of threshold across my entire stack to have some intelligence everywhere.

Agree that coherence is the change this year. That's what made the title true. I also am looking forward to Qwen 4 flash.

I guess what I'm saying is I now have an "intelligence floor" where I can have an always-on layer that can keep me from only deterministic systems to now deterministic + some amount of (now worth it) intelligence.

I wasn't really willing to do that with subscriptions or API pricing or privacy concerns.