News for hackers. Stuff that matters.
Re:Bonsai 27B: A 27B-Class Model that runs on a phone by janalsncm on Thursday July 16, 2026 @09:33PM (#48942431)
1) Mixture of experts as an LLM architecture is not the same thing. In MoE each “expert” can activate for any given token.

2) Bitter lesson is misunderstood. Specialization and inductive biases still matter. ChatGPT isn’t the best chess player in the world just because it’s seen more math problems or read more Japanese poetry. Stockfish is, because it bakes in useful inductive biases like minimax.

There are no comments on this comment yet.