News for hackers. Stuff that matters.
Re:Word Embeddings: Past, Present, and Future by orthoganol on Monday June 12, 2017 @03:40AM (#14535677)
> Instead, each word should have a vector based on context, for disambiguation and nuance.

You do realize even under vanilla Word2vec, each word embedding is a sum of both the word vector and a vector of a surrounding window of words? Each embedding results from maximizing the likelihood of a small surrounding context given the word. Thus, context is factored in.

Re:Word Embeddings: Past, Present, and Future by nl on Monday June 12, 2017 @08:28AM (#14536725)
This is true, but the context implicit in the vector is the average of the training data.

Vanilla Embedding based models don't handle word order well (as some of the examples upon this slide deck show).