Re:Demystifying Word2Vec
by twelfthnight on Tuesday February 07, 2017 @06:42PM
(#13594178)
True, you could use the right matrix of the SVD on the term document matrix. As far as I know, the embeddings won't have the same interpretation as the left matrix from SVD on the word-context matrix. (By LSA I mean SVD on the term document matrix).
Those are excellent papers, by the way.
The following two papers also go into detail about the mathematical similarities between LSA and neural embeddings and achieving similar performance with both:
Levy, O. and Goldberg, Y. (2014). Neural word embedding as implicit matrix factorization. https://www.cs.bgu.ac.il/~yoavg/publications/nips2014pmi.pdf
Levy, O., Goldberg, Y., and Dagan, I. (2015). Improving distributional similarity with lessons learned from word embeddings. http://www.anthology.aclweb.org/Q/Q15/Q15-1016.pdf