News for hackers. Stuff that matters.

A comment on Speculative KV coding: losslessly compressing KV cache by up to ~4×. Parent comment

Re:Speculative KV coding: losslessly compressing KV cache by... by lostmsu on Tuesday June 09, 2026 @12:43PM (#48463514)
27B does, the op was talking about consumer use of DS v4 Flash, that's 160GB.
[ Parent | on HN ]
Re:Speculative KV coding: losslessly compressing KV cache by... by zozbot234 on Wednesday June 10, 2026 @10:39AM (#48476982)
It has been quantized to 80GB (2-bit quantization for experts) with limited degradation. Certainly competitive with a 27B model, and especially useful in a size range where few "native" models exist.
[ Parent | on HN ]
Re:Speculative KV coding: losslessly compressing KV cache by... by lostmsu on Wednesday June 10, 2026 @11:10AM (#48477555)
> (2-bit quantization for experts) with limited degradation. Certainly competitive with a 27B model

Uh-huh...

[ Parent | on HN ]