News for hackers. Stuff that matters.

A comment on Scientists See Promise in Deep-Learning Programs. Parent comment

Re:Scientists See Promise in Deep-Learning Programs by iskander on Saturday November 24, 2012 @02:23PM (#4826169)
You can't train fully connected deep models with backprop, or at least not easily or well. An alternative solution to this problem is spatial weight pooling (Yann's convolutional networks) which play well with SGD.
[ Parent | on HN ]
Re:Scientists See Promise in Deep-Learning Programs by mark_l_watson on Saturday November 24, 2012 @03:21PM (#4826333)
That is correct. The problem is that the gradients get smaller and smaller as you back propagate back towards the input layer. So learning on the front part of the net is slow. Hinton has a lot of good material about htis in his Coursera lectures.
[ Parent | on HN ]
Re:Scientists See Promise in Deep-Learning Programs by wookietrader on Saturday November 24, 2012 @03:32PM (#4826364)
Yes you can.

Check out the publications by Ciresan on MNIST, have a look at Hinton's dropout paper or at the Kaggle competition that used deep nets. Or try it yourself and spend a descent amount of time on hyper parameter tuning. :)

[ Parent | on HN ]
Re:Scientists See Promise in Deep-Learning Programs by iskander on Saturday November 24, 2012 @08:10PM (#4826976)
Which of Ciresan's projects are you referring to? Everything I've seen by him uses convolutional layers of some sort.
[ Parent | on HN ]