Re:Learning Word Vectors for 157 Languages
by bllguo on Friday February 23, 2018 @02:49PM
(#16448838)
that's a nice link but the excerpt you quote is kind of misleading; the word vectors in this case are not one-hot encodings. They are learned, continuous representations. But one-hot representations are also a kind of word vector.
word vectors are vector representations of each word in the vocabulary. Here they are learned by a neural net. the length of the vector is the # of features. Just for intuition, one feature of a word the NN could learn is the gender of a word, and so on.
Re:Learning Word Vectors for 157 Languages
by Choco31415 on Friday February 23, 2018 @03:03PM
(#16448998)
To help with intuition, here are a few example word vectors that we might encounter:
"fruit": {food: 0.99, gender: -0.05, size: 0.2}
"king": {food: -0.9, gender: 0.92, size: 0.56}
Building off of what v1n337 stated, though, axis can easily be skewed and rotated such that they're still interpretable, just not obviously so.
Re:Learning Word Vectors for 157 Languages
by v1n337 on Friday February 23, 2018 @03:00PM
(#16448960)
Yes, quite correct! The vectors are dense representations.
But the features aren't individually interpretable, in practice. For instance, the 'gender' of a word may have it's signal scattered over several features/dimensions of the learned vector.
If you are interested in more, check out these excellent reviews by Adrian Colyer posted in The Morning Paper.
https://blog.acolyer.org/2016/04/21/the-amazing-power-of-wor...