News for hackers. Stuff that matters.

A comment on Why do we use word embeddings in NLP?. Parent comment

Re:Why do we use word embeddings in NLP? by DoctorOetker on Wednesday September 11, 2019 @02:05PM (#20942975)
I agree, and I have an idea for this dimensionality reduction which makes the original unsupervized word vectors interpretable.

it boggles my mind I haven't seen anyone implement my idea.

[ Parent | on HN ]
Re:Why do we use word embeddings in NLP? by aglionby on Thursday September 12, 2019 @08:03AM (#20950458)
SVD has been used for dimensionality reduction of co-occurrence matrices for ages [1], but the resulting word embeddings aren't as performant as those of word2vec/etc. The same is probably true of using PCA.

Word2vec's popularity is the result of people valuing performance (i.e. accuracy) more than interpretability.

[1] https://dl.acm.org/citation.cfm?id=148132

[ Parent | on HN ]
Re:Why do we use word embeddings in NLP? by gattilorenz on Wednesday September 11, 2019 @05:36PM (#20945002)
Well, it might be because it's hard to read your mind from here.
[ Parent | on HN ]
Re:Why do we use word embeddings in NLP? by DoctorOetker on Thursday September 12, 2019 @04:11AM (#20949155)
no, that wouldn't be mindboggling

what's mindboggling to me is that I haven't seen anyone else come up with the idea independently.

[ Parent | on HN ]
Re:Why do we use word embeddings in NLP? by DoctorOetker on Wednesday September 11, 2019 @07:02PM (#20945733)
it's so obvious one wouldn't have to read my mind, it's all implicit in the king-man+woman=queen type of relations... if you really ask a second time, fuck it, im not in ML sector, perhaps I just give away the idea...
[ Parent | on HN ]