News for hackers. Stuff that matters.

A comment on Bonsai 27B: A 27B-Class Model that runs on a phone.

Re:Bonsai 27B: A 27B-Class Model that runs on a phone by erwan577 on Tuesday July 14, 2026 @03:38PM (#48911989)
The KV-cache memory usage also seems remarkably frugal, even at the full context length. That could make this model particularly useful in multi-agent coding workflows.

I wish KV-cache memory usage and related optimizations were discussed more clearly in new model announcements and demos.

[ Parent | on HN ]
Re:Bonsai 27B: A 27B-Class Model that runs on a phone by verdverm on Tuesday July 14, 2026 @04:13PM (#48912393)
quanting kv cache hurts attention / recall, and long-form tasks by proxy. Model families and sizes have different tolerances to quant ting different parts of the model, same for intended tasks.
[ Parent | on HN ]