SVD has been used for dimensionality reduction of co-occurrence matrices for ages [1], but the resulting word embeddings aren't as performant as those of word2vec/etc. The same is probably true of using PCA.

Word2vec's popularity is the result of people valuing performance (i.e. accuracy) more than interpretability.

[1] https://dl.acm.org/citation.cfm?id=148132