News for hackers. Stuff that matters.

A comment on Using k-d trees to efficiently calculate nearest neighbors in 3D vector space. Parent comment

Re:Using k-d trees to efficiently calculate nearest neighbor... by _wmd on Sunday February 11, 2018 @02:24AM (#16351626)
I was toying with compqring similarity of texts just last night, is there anything that could handle 10k+ sized sparse vectors? Obviously there are tools like lucene, but I wanted to mess around in the interactive interpreter
[ Parent | on HN ]
Re:Using k-d trees to efficiently calculate nearest neighbor... by agibsonccc on Sunday February 11, 2018 @03:08AM (#16351729)
Check out LargeVis then. The original github repo by the authors has a sparse mode: https://github.com/lferry007/LargeVis/

TSNE didn't scale well no matter what you did to it (hence the newer technique) - in general though you can't beat cosine similarity of TFIDF vectors as a baseline.

[ Parent | on HN ]