News for hackers. Stuff that matters.
Re:Word vectors are awesome but you don’t need a neural netw... by arrmn on Wednesday October 18, 2017 @07:26PM (#15504178)
What exactly do you mean with "they let the models train their own word embeddings", can you elaborate more on this or are there any current papers about this topic?
Re:Word vectors are awesome but you don’t need a neural netw... by make3 on Wednesday October 18, 2017 @07:33PM (#15504220)
the embedding layer is the layer that converts the one hot word feature in to a continuous multi dimensional vector that the deep net can learn with. they used to pretrain that layer separately with word2vec. now as it's just a neural net layer, they let the translation model train it with backprop on the main (translation /dialog / qa, etc) task as a regular layer