News for hackers. Stuff that matters.
Re:Geoffrey Hinton: Introduction to Deep Learning, Deep Beli... by bradneuberg on Monday December 14, 2015 @06:56PM (#10734911)
It's not just ReLUs that have helped, its also better random initialization before starting training, such as using Xavier initialization (http://andyljones.tumblr.com/post/110998971763/an-explanatio...)

Also, batch normalization helps with convergence as well. In addition, LSTMs work when dealing with recurrent neural nets.

There are no comments on this comment yet.