News for hackers. Stuff that matters.
Re:TurboQuant: Redefining AI efficiency with extreme compres... by macleginn on Wednesday March 25, 2026 @07:14AM (#47515814)
"TurboQuant proved it can quantize the key-value cache to just 3 bits without requiring training or fine-tuning and causing any compromise in model accuracy" -- what do each 3 bits correspond to? Hardly individual keys or values, since it would limit each of them to 8 different vectors.
Re:TurboQuant: Redefining AI efficiency with extreme compres... by jbellis on Wednesday March 25, 2026 @07:42AM (#47516019)
The explanation is terrible, but it's clear that it's not actually lossless.
Re:TurboQuant: Redefining AI efficiency with extreme compres... by carlosvega on Wednesday March 25, 2026 @08:40AM (#47516550)
Is the number of bits per coordinate. So, 1 bit is 2x2 grid. 3 bit is a 64 cell grid (2^3 x 2^3). Here you have a demo.

https://mesuvash.github.io/blog/2026/turboquant-interactive/