Text Matching Using Cosine Similarity

Re:Text Matching Using Cosine Similarity by 0xfaded on Thursday December 27, 2018 @11:42AM (#18770542)

Keep in mind that any similarity measure is mapping a high dimensional space onto a one dimensional space. The volume of vectors within a cosine of another vector is an N dimensional cone with volume rapidly decaying towards 0 for N > 5 (assuming positive cosine). Therefore cosine is not a particularly good metric in high dimensional spaces.

[ Parent | on HN ]

Re:Text Matching Using Cosine Similarity by SubiculumCode on Thursday December 27, 2018 @02:44PM (#18771971)

For n ~ 250k to 1.5M, I've use Manhattan distance (as opposed to Euclidean) in an analysis of neuroimaging functional data...but I am interested in people's takes on choosing a distance metric..There are so many exotics out there.

[ Parent | on HN ]

Re:Text Matching Using Cosine Similarity by nixpulvis on Thursday December 27, 2018 @04:24PM (#18772764)

What if my model is really good and maps to the angle between two vectors almost perfectly?

[ Parent | on HN ]

Re:Text Matching Using Cosine Similarity by ivalm on Thursday December 27, 2018 @01:07PM (#18771211)

Kind of, w2v is cosine similarity driven and there embeddings often go up to 300 dimensions.

[ Parent | on HN ]