Re:Is Cosine-Similarity of Embeddings Really About Similarity?
by pletnes on Tuesday March 12, 2024 @01:39AM
(#39676499)
I sure hope noone claimed that. You’re doing potentially huge dimensionality reduction, uniqueness would be like saying you cannot have md5 collisions.
[ Parent
| on HN ]
Re:Is Cosine-Similarity of Embeddings Really About Similarity?
by JohnKemeny on Tuesday March 12, 2024 @04:43AM
(#39677396)
Johnson and Lindenstrauss disagree. https://en.wikipedia.org/wiki/Johnson%E2%80%93Lindenstrauss_...
Re:Is Cosine-Similarity of Embeddings Really About Similarity?
by yorwba on Tuesday March 12, 2024 @05:34AM
(#39677632)
If you have 1000 points and want to preserve their squared distances to within an error of 1%, the Johnson-Lindenstrauss construction suggests an embedding dimension of 8(ln 1000)/(0.01²) > 552620. If your points start out in a lower-dimensional space than that to begin with, it's obviously pointless.
The crossover point where the number of dimensions falls below the number of points is at 1113868. If you're willing to tolerate 10% error, it's at 7094.