word2vec in yhat: Word vector similarity

Re:word2vec in yhat: Word vector similarity by gojomo on Monday September 30, 2013 @02:30PM (#6471342)

I'm still impressed it only takes 26 letters, in words of average size around 5! By comparison, 1000 continuous dimensions seems positively resplendent with expressiveness.

FWIW, 2^61 > 26^5, so even the binary vector 2^1000 has an expressive space about 2^939 times larger than 26^5 (all possible words up to 5 letters).

[ Parent | on HN ]

Re:word2vec in yhat: Word vector similarity by judk on Monday September 30, 2013 @04:10PM (#6472044)

Yes, but there are exponentially more concepts than words. The words we have are sparse set of labels for particularly relevant combinations.

But yeah, the continuous dimensions can hide many more binary dimensions.

For example, 4-D rgba can be smashed into 1 continuous (or 64-bit) dimension, but that feels a bit like cheating.

So it sort of feels like 1000 64-bit dimensions is a tricky name. 64000 1bit dimensions.

[ Parent | on HN ]