News for hackers. Stuff that matters.

A comment on The Theoretical Limitations of Embedding-Based Retrieval. Parent comment

Re:The Theoretical Limitations of Embedding-Based Retrieval by CuriouslyC on Friday August 29, 2025 @08:26PM (#45070869)
If you consider the actual latent space the full higher dimensional representation, and you take the first principle component, the other vectors are zero. Pretty sparse. No it's not a linked list sparse matrix. Don't be a pedant.
[ Parent | on HN ]
Re:The Theoretical Limitations of Embedding-Based Retrieval by zwaps on Saturday August 30, 2025 @02:44AM (#45072427)
No one means Matryoshka embeddings when they talk about sparse embeddings. This is not pedantic.
[ Parent | on HN ]
Re:The Theoretical Limitations of Embedding-Based Retrieval by CuriouslyC on Saturday August 30, 2025 @02:48AM (#45072445)
No one means wolves when they talk about dogs, obviously wolves and dogs are TOTALLY different things.
[ Parent | on HN ]
Re:The Theoretical Limitations of Embedding-Based Retrieval by cap11235 on Saturday August 30, 2025 @09:56AM (#45074730)
Why?
[ Parent | on HN ]
Re:The Theoretical Limitations of Embedding-Based Retrieval by yorwba on Saturday August 30, 2025 @02:55AM (#45072485)
When you truncate Matryoshka embeddings, you get the storage benefits of low-dimensional vectors with the limited expressiveness of low-dimensional vectors. Usually, what people look for in sparse vectors is to combine the storage benefits of low-dimensional vectors with the expressiveness of high-dimensional vectors. For that, you need the non-zero dimensions to be different for different vectors.
[ Parent | on HN ]