We found sparse, truncated PCA to be the most competitive baseline. We beat it by a lot (see the paper [1]), but the other big drawback is that trading off the rank vs sparsity was an ugly hyperparameter tuning problem. By ugly, I mean that the results were really sensitive to getting this right, it wasn't easy to set a priori, and took a while to iterate on because the sparse PCA trained much more slowly than any other practical alternative.

There are situations where PCA/SVD is the right approach though. Namely, if you need really little error, our method often can't do that, whereas throwing away dims that explain almost no variance can. Also it's just easier to implement.

[1] https://arxiv.org/pdf/2106.10860.pdf

SVD performs poorly for compression in terms of accuracy / compression ratio. As the Bolt paper said, for that, product quantization is the way to go. However, the accuracy trade-off is still pretty big, that's why it is mostly used for coarse similarity recalls (such as faiss or scann).