Why do we use word embeddings in NLP?

Re:Why do we use word embeddings in NLP? by ecaradec on Tuesday September 10, 2019 @09:43AM (#20928066)

What you describe is word2vec which is one kind of word embedding that is pre-learned. Often (with keras at least ), embedding is learned simultaneously with the deep learning network. Keras uses index of words to progressively adjust a set of randomly initialized n-dimensional vectors. I’m telling you because I feel it may be the case for you too. The various kind of embedding was very confusing at first.

However, I just marvel at word2vec when I stumbled upon it. Encoding of meaning as vector dimensions was mind expanding for me.

[ Parent | on HN ]

Re:Why do we use word embeddings in NLP? by elephant_7 on Tuesday September 10, 2019 @10:21AM (#20928496)

These guys laid a lot of the groundwork for embeddings before word2vec while also showing practical applications in finance https://www.elastic.co/blog/generating-and-visualizing-alpha...

[ Parent | on HN ]

Re:Why do we use word embeddings in NLP? by aglionby on Tuesday September 10, 2019 @11:44AM (#20929442)

It's also possible to initialise a word embedding matrix with vectors trained with word2vec (or any other type of pre-trained embeddings: fastText, GLoVe being common) to get a perf boost. Of course, you have to ensure the embedding matrix is pre-populated with respect to the word-index mapping. Then, as you describe, these can be adjusted during training, although sometimes they're fixed. The boost is more noticeable when you're training a model on small amounts of data.

[ Parent | on HN ]

Re:Why do we use word embeddings in NLP? by donretag on Tuesday September 10, 2019 @10:30AM (#20928607)

For me, the marvel of word2vec is that we do not need to embedding meaning into words. The move toward semantic understanding and meaning is not essential, but this post think that a semantic representation is the next logical step.

[ Parent | on HN ]