News for hackers. Stuff that matters.
Re:How does cosine similarity work? by magicalhippo on Friday September 06, 2024 @06:16PM (#41470114)
I've been wondering the same.

When I dabbled with latent semantic indexing[1], using cosine similarity made sense as the dimensions of the input vectors were words, for example a 1 if a word was present or 0 if not. So one would expect vectors that point in a similar direction to be related.

I haven't studied LLM embedding layers in depth, so yeah been wondering about using certain norms[2] instead to determine if two embeddings are similar. Does it depends on the embedding layer for example?

Should be noted it's been many years since I learned linear algebra, so getting somewhat rusty.

[1]: https://en.wikipedia.org/wiki/Latent_semantic_analysis

[2]: https://en.m.wikipedia.org/wiki/Norm_(mathematics)

There are no comments on this comment yet.