News for hackers. Stuff that matters.

A comment on The Unreasonable Effectiveness of Recurrent Neural Networks.

Re:The Unreasonable Effectiveness of Recurrent Neural Networks by Patryk on Friday May 22, 2015 @12:27AM (#9586565)
This same thing (i.e., using recurrent neural networks to predict characters (and even words)) was done by Elman in 1990 in a paper called "Finding Structure in Time"[1]. In that paper, Elman goes several steps further and carries out some analysis to show what kind of information the recurrent neural network maintains about the on-going inputs.

It's an excellent read for anyone interested in learning about recurrent neural networks.

[1] http://crl.ucsd.edu/~elman/Papers/fsit.pdf

[ Parent | on HN ]
Re:The Unreasonable Effectiveness of Recurrent Neural Networks by sushirain on Sunday May 24, 2015 @04:54PM (#9597552)
It's amazing how much was already known decades ago. Elman and others did much more, and hopefully, now the field will take the next step (which was long delayed), with the help of today's computer power.
[ Parent | on HN ]