News for hackers. Stuff that matters.
Re:TurboQuant: Redefining AI efficiency with extreme compres... by spencerflem on Wednesday March 25, 2026 @03:11AM (#47514250)
I like the visualization, but I don’t understand the grid quantization. If every point is on the unit circle aren’t all the center grid cords unused?
Re:TurboQuant: Redefining AI efficiency with extreme compres... by vincnetas on Wednesday March 25, 2026 @03:39AM (#47514433)
i think grid can be a surface of the unit sphere
Re:TurboQuant: Redefining AI efficiency with extreme compres... by fc417fc802 on Wednesday March 25, 2026 @08:49AM (#47516651)
Yeah that's odd. It seems like you'd want an n-1 dimensional grid on the surface of the unit sphere rather than an n dimensional grid within which the sphere resides.

Looking at the paper (https://arxiv.org/abs/2504.19874) they cite earlier work that does exactly that. They object that grid projection and binary search perform exceptionally poorly on the GPU.

I don't think they're using a regular grid as depicted on the linked page. Equation 4 from the paper is how they compute centroids for the MSE optimal quantizer.

Why specify MSE optimal you ask? Yeah so it turns out there's actually two quantization steps, a detail also omitted from the linked page. They apply QJL quantization to the residual of the grid quantized data.

My description is almost certainly missing key details; I'm not great at math and this is sufficiently dense to be a slog.

Re:TurboQuant: Redefining AI efficiency with extreme compres... by mesuvash on Wednesday March 25, 2026 @09:16PM (#47525592)
Yes. Great catch. I simplified the grid just for visualization purpose.

I've updated the visualization. The grid is actually not uniformly spaced. Each coordinate is quantized independently using optimal centroids for the known coordinate distribution. In 2D, unit-circle coordinates follow the arcsine distribution (concentrating near ±1), so the centroids cluster at the edges, not the center.

Re:TurboQuant: Redefining AI efficiency with extreme compres... by spencerflem on Thursday March 26, 2026 @10:07AM (#47530653)
Cool! Thank you