News for hackers. Stuff that matters.
Re:Why do we use word embeddings in NLP? by yorwba on Tuesday September 10, 2019 @11:18AM (#20929163)
The contextualized word embeddings you get out of BERT are still generated from fixed per-word vectors. And while you get one output vector for each input vector, that doesn't mean they correspond to each other. The model could arbitrarily reshuffle information between outputs, so long as the output as a whole reflects the input sufficiently well. So BERT embeddings are not "word embeddings" in the usual sense.

There are no comments on this comment yet.