News for hackers. Stuff that matters.
Re:Bonsai 27B: A 27B-Class Model that runs on a phone by verdverm on Tuesday July 14, 2026 @06:18PM (#48913645)
I haven't dug into QAT deeply, better recovery is my understanding as well, and also that it is out of reach for most people because you have to train a model to back prop errors based on estimated error under quant.

Hopefully more of the lab releases are trained under QAT so we can all benefit.

Re:Bonsai 27B: A 27B-Class Model that runs on a phone by dofm on Tuesday July 14, 2026 @06:59PM (#48914014)
I think they did Gemma 3 QAT models and there are QAT versions of essentially all the Gemma 4 models (including DiffusionGemma).