Re:Why Deep Learning Cannot Be Applied to Natural Languages ...
by lucidrains on Monday January 02, 2017 @04:22PM
(#13304662)
this paper still consistently blows my mind whenever it is brought up. the future is clearly headed for complete end-to-end learning systems, maybe even with learning to learn methods. https://youtu.be/x1kf4Zojtb0?t=1h4m54s
In general, RNNs are especially fit for discrete sequences. And their continuous representation is actually an advantage (so they can see that two words are similar or analogous). BTW: see my draft on word2vec: http://p.migdal.pl/2016/12/30/why-do-word2vec-analogies-work...