News for hackers. Stuff that matters.
Re:Honey, I shrunk the embeddings: Matryoshka vs. PCA by stephantul on Sunday August 09, 2026 @10:32AM (#49231804)
Nice! I’ve been working on something similar and found similar results.

In my experiments, I used lots of embedding models and the results were not nearly as uniform as this curve, just FYI. I didn’t use any of the API-based models though

I also wrote about this exact comparison when using PCA and MRL to quantize static models, see: https://stephantul.github.io/blog/mrl-pca/

Re:Honey, I shrunk the embeddings: Matryoshka vs. PCA by dcastm on Sunday August 09, 2026 @11:49AM (#49232517)
Thank you! Will take a look at your results.

I couldn't find much when I first looked into this, which is why I ended up writing the article.

Re:Honey, I shrunk the embeddings: Matryoshka vs. PCA by djoldman on Sunday August 09, 2026 @12:16PM (#49232754)
Did you find much difference in inference latency or throughput between baseline and PCA?
Re:Honey, I shrunk the embeddings: Matryoshka vs. PCA by stephantul on Sunday August 09, 2026 @12:29PM (#49232909)
PCA is applied after the model, so there should be no difference in embedding throughput. Lookups in the index should be faster, but that speedup also applies equally to MRL.

So I guess the answer is: no

Re:Honey, I shrunk the embeddings: Matryoshka vs. PCA by stephantul on Sunday August 09, 2026 @11:51AM (#49232530)
Ah I meant more to say that I was working on this as well. I haven’t published the results for this comparison specifically yet.