Andrej Karpathy's blog is excellent as a first introduction. Start with http://karpathy.github.io/neuralnets/

He also has a fantastic article specifically about recurrent neural networks here: http://karpathy.github.io/2015/05/21/rnn-effectiveness/

Thanks! I'm really enjoying seeing the huge variety of domains people have applied char-rnn to. I haven't anticipated response at this scale, to both the original blog post and the code release. (I try to keep a somewhat up to date list of responses on the bottom of the blog - cooking recipes, music, obama speeches, magic cards...)

In terms of learning, I would also encourage people to try the materials from our CS231n class - this is a class I taught at Stanford last quarter with my adviser. It's technically about Convolutional Networks, but most of the class material is building up generic Neural Networks, backprop, and so on. You can also try our IPython Notebook assignments.

Course notes: http://cs231n.github.io/ Syllabus with slides too: http://cs231n.stanford.edu/syllabus.html

Another good pointer is Andrew Ng's Coursera class - that's a thorough introduction as well.

Awesome, thanks for this! I started reading the first link and I'm hooked, that's some extremely interesting stuff

Keras http://keras.io/ is a neural networks library for Python that is rather easy to get started with compared to the alternatives.

I also recommend the tutorials on http://deeplearning.net/tutorial/ (Python / Theano)

And Andrej Karpathy's blog is also a great resource for explanations of deep learning concepts in simple terms: https://karpathy.github.io/

I've found http://neuralnetworksanddeeplearning.com by Michael Neilson to be an excellent resource.

This is the one I've always recommended as well. It's a little longer, but very in-depth.

Awesome, will look into, Thanks!

Matlab has a neural network toolbox that you can use to train small recurrent networks.

There is a book that I can link if you are interested.

I am interested :)

This is from one of the people who helped write the toolbox in Matlab. There is a page limit on the Amazon publishing service, so that is why it's posted here[1]. Chapter 14 describes notation for the recurrent networks and explains their training.

[1]:http://hagan.okstate.edu/NNDesign.pdf

If you are comfortable in MATLAB this tutorial is great: https://theclevermachine.wordpress.com/2014/09/11/a-gentle-i...