News for hackers. Stuff that matters.
Re:Is Cosine-Similarity of Embeddings Really About Similarity? by soarerz on Tuesday March 12, 2024 @12:08AM (#39676109)
What is the cheapest way to capture similarity if not via dot product then?
Re:Is Cosine-Similarity of Embeddings Really About Similarity? by gajus on Tuesday March 12, 2024 @01:06AM (#39676362)
Interested to know as well
Re:Is Cosine-Similarity of Embeddings Really About Similarity? by latency-guy2 on Tuesday March 12, 2024 @02:27AM (#39676723)
I don't have an answer for this really outside of silly ones like "strict equality check", but I assert that no one else does either, at least today and right now, and its an inherent limitation due to the nature of embeddings and the space it desires to be (cheap, fast, good enough similarity for your use case).

You're probably best off using the commercial suggestion, and if its dot product, go for it. I am no expert in this area and my interest wanes every day.

Re:Is Cosine-Similarity of Embeddings Really About Similarity? by scotty79 on Tuesday March 12, 2024 @04:49AM (#39677422)
Instead of sums of multiplications you could for example use sum of squares of differences.

Means squared error instead of dot product, it's not cheaper but it's close

If you want to go cheaper you could use sum of abs of differences.

Re:Is Cosine-Similarity of Embeddings Really About Similarity? by soarerz on Wednesday March 13, 2024 @10:07AM (#39691366)
This is effectively "the same" as dot product.

For a lot of embeddings we have today, norm of any embedding vector is roughly of same size, so the angle between two vectors is roughly same size as length of difference that you are saying, and can be expressed in terms of 1 - dot product after scaling