Re:Cosine Similarity Isn't the Silver Bullet We Thought It Was
by refulgentis on Thursday January 16, 2025 @06:09PM
(#42732090)
Flagged. Significant parts AI-generated, and it misrepresents the writeup. It is titled "Cosine Similarity Isn't the Silver Bullet We Thought It Was", and the article is (editorializing) "it's a bad idea to use LLM embeddings, trained for dot product, for cosine similarity"
From paper:
- "when a model is trained w.r.t. the dotproduct, its effect on cosine-similarity can be opaque and sometimes not even
unique. One solution obviously is to train the model w.r.t. to cosine similarity"
- "We find that the underlying reason is not cosine similarity itself, but the fact that the learned embeddings have a degree of freedom
that can render arbitrary cosine-similarities even though their (unnormalized)
dot-products are well-defined and unique"
From paper:
- "when a model is trained w.r.t. the dotproduct, its effect on cosine-similarity can be opaque and sometimes not even unique. One solution obviously is to train the model w.r.t. to cosine similarity"
- "We find that the underlying reason is not cosine similarity itself, but the fact that the learned embeddings have a degree of freedom that can render arbitrary cosine-similarities even though their (unnormalized) dot-products are well-defined and unique"