Re:Word Embeddings: Past, Present, and Future
by visarga on Monday June 12, 2017 @01:16AM
(#14535309)
I realized after posting that I was naive. LSTMs are practically word embedding combinators. Training LSTMs should solve the problem of combining words and interpreting them in context.
Re:Word Embeddings: Past, Present, and Future
by Jack000 on Monday June 12, 2017 @02:45AM
(#14535521)
I thought Spacy only did part of speech disambiguation rather than sense disambiguation. ie. it would be confused if a word has multiple meanings, all of which are nouns.
This demo uses the Spacy POS tagging to build a word embedding which disambiguates multiple senses. The paper explains it: https://arxiv.org/abs/1511.06388
For many downstream tasks it doesn't seem to improve things though. NLP is hard - lots of things should work but don't because of the sparse training data. Like this slide deck says: most sentences occur only once.