Re:Improving the way neural networks learn
by araes on Friday August 01, 2014 @03:22PM
(#8122291)
It seems like the implicit target in the document is to achieve a critically damped system with no ringdown on learning. However, if they're trying to go for speed, then it seems like they should accept possible overshoot, and use non-linear control theory for their weights so that they're underdamped during the initial descent, and then transition into critically damped gradient descent as they move into the flat zone. Something like a variable "damper" or weights/springs based on current error. Perhaps that is done elsewhere though, and just not described as a technique here.
[ Parent
| on HN ]
Re:Improving the way neural networks learn
by Houshalter on Saturday August 02, 2014 @12:26AM
(#8123997)
That sort of sounds like Rprop which is supposedly the fastest learning algorithm. There are also other "adaptive learning rate" algorithms.