Re:Word embeddings in 2017: Trends and future directions
by PaulHoule on Saturday October 21, 2017 @08:22PM
(#15524401)
Maybe I'm not reading it right, but that arXiv paper about CoVE doesn't seem to be getting anywhere near commercially useful results.
For instance, the random result for ImDB in Table 2 is 88.4 and the best one is 92.1; that's really not a lot of lift. I could see TREC-6 and TREC-50 results being good enough to let off the leash, but I still have a hard time picturing this being useful in the real world.
It's not word embeddings job to handle phrases - but nearly all modern phrase embedding algorithms sit on top of word embeddings. They often create a weighted average of embeddings by using an attention model, or they can use a more complex model such as an LSTM with attention (e.g. CoVE - https://arxiv.org/abs/1708.00107 ).
Word embeddings can handle polysemy - high dimensional vectors can (and do) hold information of various types that is used in different contexts in different ways. Some approaches deal with this more directly (e.g. including part-of-speech as part of the vocab item), and that sometimes can help a bit.