Re:Show HN: Word2Bits – Quantized Word Vectors
by maxlam on Tuesday March 20, 2018 @04:08AM
(#16626729)
Definitely tried to figure out if the dimensions mean anything -- as far as I can tell they don't really mean much :(
Re:Show HN: Word2Bits – Quantized Word Vectors
by Maybestring on Tuesday March 20, 2018 @12:18PM
(#16629933)
If you want them to be meaningful without changing the model...
Couldn't you rotate the basis to minimize the distance between each basis vector and it's nearest neighbor?
Re:Show HN: Word2Bits – Quantized Word Vectors
by maxlam on Tuesday March 20, 2018 @04:56PM
(#16632560)
Haven't tried this but this is definitely a good idea for visualizing what's going on!
Re:Show HN: Word2Bits – Quantized Word Vectors
by loxias on Tuesday March 20, 2018 @02:00PM
(#16630952)
One of the fascinating things about neural embedding such as these is that the individual component dimensions have NO "real" semantic meaning to us humans. It's better to think of them as single points in a higher dimensional space.
(of course, with a clever network design you could probably FORCE "meaning" onto some components)
Also, I am curious why you chose to go straight to publishing on Arxiv? I am actually also in CS224N right now and have a project me and my collaborator feel is publication worthy, but our plan is to go the normal route of submitting to a conference and only putting it on Arxiv after the review process (though our code is open source, so now that I think about it maybe that's not that useful a plan...).