Re:Understanding Deep Learning Through Neuron Deletion
by LoSboccacc on Friday March 23, 2018 @07:29AM
(#16656916)
I've lost the original files, I digitalized some of the important part back, here https://imgur.com/a/w2BCy
it was 2004-2006, we used weight quantization as noise function and a modified levemberg-marquad for training, matlab was the hot stuff back then and by today standard everything is ancient, so there's that - the training alghorithm was changed to train toward a flatter area of the noise function, so it used the second derivative to hold gradient descent and overfitting.
Re:Understanding Deep Learning Through Neuron Deletion
by pilooch on Friday March 23, 2018 @10:47AM
(#16658678)
Many thanks for the thorough answer! Will read and ask questions if any. We have customers we help quantize the models for their FPGAs and this may be an avenue we could test.
it was 2004-2006, we used weight quantization as noise function and a modified levemberg-marquad for training, matlab was the hot stuff back then and by today standard everything is ancient, so there's that - the training alghorithm was changed to train toward a flatter area of the noise function, so it used the second derivative to hold gradient descent and overfitting.
I can answer specific questions if you want.