Re:Organizing My Emails with a Neural Net
by andreyk on Saturday January 23, 2016 @09:23PM
(#10961178)
I ran one experiment with an Embedding layer (second one in the Deep Learning Is No Good Here section), which I assumed to be analagous to word2vec. I hoped it would help as well so was rather dissapointed it did not seem to be so. Perhaps I should have tried a few more configurations, though.
[ Parent
| on HN ]
An Embedding layer by itself will try to learn word vectors from the data you train it on (error gradient will propagate back to embedding layer, update the vector weights). Word2vec and word vectors are only really useful with tons of training data to learn good embeddings. I believe OP is referring to using Google's pre-trained word-vectors (trained on massive amounts of text [100 Billion words]). Can be found here: https://code.google.com/p/word2vec/
This is pretty straightforward to implement in keras, you just need to supply pre-trained word-vectors weights to your embedding layer.
Embedding(vocab_size, 300, weights = [word2vec_weights])
Re:Organizing My Emails with a Neural Net
by andreyk on Sunday January 24, 2016 @12:49PM
(#10963221)
Ah, I honestly assumed it would use an existing set of weights such as from word2vec. Though I now realize that does not make sense since you can also embed images or really any real numbered inputs. I will give this a try.