A Gentle Introduction to Backpropagation

Re:A Gentle Introduction to Backpropagation by jbarrow on Tuesday August 05, 2014 @11:36AM (#8137460)

It's true that the beginning of the talk is about how people abandoned backprop trained neural networks because they often underperformed SVMs, but the rest of the talk is about deep learning, which introduced a new generation of neural nets (since 2006) that are the current state of the art for a lot of problems.

In fact, deep neural networks are trained in an unsupervised manner at first, but then back propagation is used to "fine tune" and improve the results. Because they require unlabeled data sets, and can perform so well, research into neural networks has experienced a recent resurgence.

By the way, any talk by Geoff Hinton is fantastic. If you are interested in neural networks and their capabilities, and you haven't already seen it, his Coursera course [1] builds from a simple linear perceptron to the current deep learning methods.

[1] https://class.coursera.org/neuralnets-2012-001 (You'll have to sign in to see it)

[ Parent | on HN ]

Re:A Gentle Introduction to Backpropagation by agibsonccc on Tuesday August 05, 2014 @12:06PM (#8137630)

I want to add to this, that there's a lot of work that doesn't require pretraining. I recently implemented the more advanced hessian free optimization methods that don't require pretraining in my deep learning framework. The results are amazing. I'm hoping to demonstrate a lot of the tradeoffs of the different methods in a more comprehensive manner here shortly. This was an updated extension by some of hinton's students. The paper I implemented was:

http://www.cs.toronto.edu/~rkiros/papers/shf13.pdf

[ Parent | on HN ]

Re:A Gentle Introduction to Backpropagation by taylorbuley on Tuesday August 05, 2014 @12:39PM (#8137804)

Thanks for the extra resources. Any more are very welcome. Neural nets sometimes feel like quite the black box, despite their ease of implementation and apparent power.

[ Parent | on HN ]