useracd
Re:Understanding word vectors
on Friday March 02, 2018 @11:50AM (#16503184)
You're correct! The keras example implementation of FastText uses a global average pooling is used to reduce the dimensionality [0].
[0] https://github.com/keras-team/keras/blob/master/examples/imd...
Re:Understanding word vectors
on Friday March 02, 2018 @11:48AM (#16503148)
Often the w2v embedding layer will be the first layer of a network, through which a document of word representations will be passed. The output of the embedding layer will be a 2 dimensional tensor with the embedding dimension in one direction and the number of input words in the other. The next step is often to apply zero or more convolutional units over the word direction before some kind of pooling to get a 1 dimensional output. The output of the pooling layer will be a document representation based upon the word vectors.
Other approaches like doc2vec are also used.