Re:Embeddings: What they are and why they matter
by simonw on Wednesday October 25, 2023 @10:26AM
(#38013172)
Sounds like you know a great deal more about the history of this field than I do!
Re:Embeddings: What they are and why they matter
by IKantRead on Wednesday October 25, 2023 @10:46AM
(#38013404)
The classic Information Retrieval, though out of date in many ways, does a great job covering the "old school" NLP approaches for IR and has an excellent section on Latent Semantic Indexing which you might find enjoyable: https://nlp.stanford.edu/IR-book/html/htmledition/latent-sem...
Latent Semantic Indexing long precedes word2vec (I believe the initial paper was 1988) but also attempted to derived semantic vectors (and was arguably successful) using SVD on the term-frequency matrix representing a corpus.
I would certainly consider this an example of using "embeddings" that was quite heavily used in practice prior to the explosion of deep learning techniques.