amitport

not at all creepy

the only reasonable applications are in showbiz. It's a movie atunt, a theme park ride.. nothing actually useful.

I've been considering something like this for a while. It only makes sense as a free, open-source, decentralized sharing protocol (where anyone can host theorems and no one can limit sharing them).

If a company were to manage to commercialize this, it would end public open research.

C# did not ship with async/await, and Java didn't have virtual threads back then. I am specifically referring to the initial choices made in C#'s foundation.

The mystery of why .NET got so many things right is simply that C# was built several years later by the exact same Microsoft engineers who had previously worked on extending Java, giving them a perfect blank slate to fix the architectural flaws they had already encountered

Second mover advantage.

"parents and grandparents,"

You don't have to go into historical events. This is still happening now.

Jews are still fighting for their survival and the moment Israel stops fighting, millions of Jews will die.

To be fair, NPM sucked long before it got acquired by Github/Microsoft.

And to be fair 2: The other package repos also suck.

Jews are only 0.2%. So assume it happens x500 than what you see in the no-Jews-no-news mainstream media.

Hi, thanks! I appreciate your input and generally agree. The TDS article wasn't really aimed at the HN crowd, but it did help a bit with the more general audience.

I do plan to also develop an interactive guide that breaks down post-rotation quantization fundamentals in a more educational, hands-on way.

For context: https://towardsdatascience.com/how-a-2021-quantization-algor...

I recently wrote a beginner-friendly explanation of this situation in TDS:

https://towardsdatascience.com/how-a-2021-quantization-algor...

Hi, I’m the author of the post above.

tl;dr

TurboQuant is a recent paper from Google and NYU that has gained massive traction in mainstream media and the AI community. As implementations of TurboQuant are integrated into various popular projects, it is important to note its relation to EDEN quantization.

TurboQuant is essentially a partial implementation of EDEN quantization (first work published in NeurIPS 2021, extention published on ICML 2022). The few differences that do exist make EDEN significantly better.

We have also published a detailed comparative report here: https://arxiv.org/abs/2604.18555

In the vLLM documentation quoted above, TurboQuant (which is a restricted version of EDEN) is referred to as a specific case of HIGGS. Note the symmetry: EDEN acts as a special case of HIGGS; hence, HIGGS functions as a generalization of EDEN.

In any case, the quantizer is indeed an extension, regardless of whether it was explicitly framed that way in the paper. I say this not to diminish their contribution at all, but just to clarify the relationship, as it was also stated in the vLLM doc.

Thanks for the pushback, and I appreciate the reference to classical information theory.

While I probably overstated things by using the very general phrase "taking advantage," I want to be very precise about the claim, as I believe these works are foundational to quantization, beyond the scope of deep learning. The mechanism of applying a deterministic biased quantizer, such as Lloyd-Max, to the induced post-rotation distribution, alongside mathematically correcting its inherent bias, is a distinct contribution (which asymptotically improves the worst-case error).

If there is a classical paper that utilizes such a combination, I would genuinely be very eager to review it. But to my knowledge, this was not introduced prior to DRIVE and EDEN.

Thanks for that!

It is worth noting that taking advantage of the post-rotation distribution was not actually done until DRIVE (2021), which was made possible via our proper scaling. Furthermore, applying a Lloyd-Max codebook post-rotation was introduced EDEN.

We consider these to be the foundational works in this regard.

Thanks for that! Note that the residual chain is empirically and theoretically inferior to our unbiased scale; furthermore, it requires an additional bit in certain cases. Additionally, TurboQuant was not the first to apply EDEN to KV-cache (see for example https://arxiv.org/abs/2411.17525 from 2024).

Those works did cite DRIVE/EDEN :)

HIGGS is an extension of EDEN (using the well known method for blockwise Lloyd-Max).

The proper framing of this "TurboQuant" layer in vllm (which does not include JQL) is precisely EDEN 22 without the scale correction.

I believe our claim at this point is more fundamental than just lack of citation.

The quantizer in TurboQuant is EDEN quantization (2021) applied to the KV-cache. It is neither a novel quantizer nor an improvement in quantization techniques.

In DRIVE/EDEN, we already introduced the version used in "TurboQuant"'s paper and suggested an optimal scale configurations which are better in both mse-minimizing and unbiased scenarios.

Thanks for the quick response and for being willing to update the explainer. I really appreciate the clarification.

When you use TurboQuant, you are essentially using the EDEN quantizer under a different name applied to KV-cache.

Both EDEN and its 1-bit variant have been implemented in PyTorch, JAX, and TensorFlow across numerous open-source libraries and are used in various applications. I am currently writing a blog post that will document these in detail.

EDEN defines a scale parameter, S, for which we suggest specific optimal values for both biased and unbiased versions. As shown in the note I shared, these values lead to clear empirical improvements. Consequently, users who rely on the less optimal S value and the unbiasing method popularized by TurboQuant will generally see inferior results compared to those using EDEN with the optimal scale values suggested in our original papers.

The note includes extensive experiments and reproduces many of the figures from the TurboQuant paper in our Section 5. Honestly, I think our case is pretty clear-cut as is. I am not sure what the overhead for those specific benchmarks would be, but we will look into it.

(In any case, I want to emphasize that TurboQuant quantizer is a private case of EDEN)

TurboQuant IS restricted EDEN quantization (NeurIPS 21, ICML 22). Missing the optimal scale derivations, which makes TurboQuant variant considerbly LESS accurate than those works. We show this throughfully in a new note in https://arxiv.org/abs/2604.18555

We were the first to introduce post-rotation distribution aware quantization in 21, which was LATER implemented in many fields including federated learning, vector retrieval, databases, inference engines, and KV-cache.

It would be nice to get some credit of this. And it is certainly baffling to see the name "TurboQuant" repeated in this context, considering the many works from 21 onwards.

The blog post above basically goes you through EDEN quantization, but then ends settling with a less than optimal MSE-minimizing version and an unbiasing trick that often costs a full bit more than DRIVE/EDEN need for the same result.

They must mean 'The Arab World Has Lost America,' don't they?

We do cite Johnson-Lindenstrauss and the paper you shared. Please read our paper to see how the rotation-aware bias correction we introduced efficiently fixes the bias and provides a better worst-case error.

To be clear, I am not claiming they stole an idea. They have made significant independent research. However, a specific part regarding the treatment of rotation with bias correction relates to prior work, and it would be appropriate to have that recognized.

In this context, the rotation is for spreading energy and ensuring predictable coordinate distributions rather than diagonalization; it makes coordinate-wise quantization much more computationally efficient, though it throws away learnable structure.

Yes, the important part is that the normalized error does not increase with the dimension of the vector (which does happen when using biased quantizers)

It is expected that bigger vectors have proportionally bigger error, nothing can be done by the quantizer about that.

This is a great development for KV cache compression. I did notice a missing citation in the related works regarding the core mathematical mechanism, though. The foundational technique of applying a geometric rotation prior to extreme quantization, specifically for managing the high-dimensional geometry and enabling proper bias correction, was introduced in our NeurIPS 2021 paper, "DRIVE" (https://proceedings.neurips.cc/paper/2021/hash/0397758f8990c...). We used this exact rotational approach and a similar bias correction mechanism to achieve optimal distributed mean estimation. I also presented this work and subsequent papers in a private invited talk at Google shortly after publication. Given the strong theoretical overlap with the mechanisms in TurboQuant and PolarQuant, I hope to see this prior art acknowledged in the upcoming camera-ready versions.