Deep Learning 101

Re:Deep Learning 101 by jules on Thursday November 14, 2013 @04:25AM (#6731446)

> And around ~2012, a bunch of researchers have reported you don't even need 2nd-derivative information. You just have to initialize the neural net properly.

This sounds very interesting. How do you property initialize the weights? Do you have a link to a paper about this?

[ Parent | on HN ]

Re:Deep Learning 101 by kkjkok on Thursday November 14, 2013 @08:57PM (#6737219)

Check out this paper:

Practical recommendations for gradient-based training of deep architectures, Y. Bengio

http://arxiv.org/abs/1206.5533

There is a section on weight initialization on page 15. In general, this paper has a lot of good information in one place.

[ Parent | on HN ]