Re:How does cosine similarity work?
by Scene_Cast2 on Friday September 06, 2024 @07:18PM
(#41470392)
Well, depends. For some models (especially two tower style models that use a dot product), you're definitely right and it makes a huge difference. In my very limited experience with LLM embeddings, it doesn't seem to make a difference.
[ Parent
| on HN ]
Interesting, I hadn’t heard of two tower modes before!
Yes, I guess it’s curious that the information lost doesn’t seem very significant (this also matches my experience!)
Re:How does cosine similarity work?
by Scene_Cast2 on Friday September 06, 2024 @11:05PM
(#41471219)
Two tower models (and various variants thereof) are popular for early stages of recommendation system pipelines and search engine pipelines.