Understanding Deep Learning Through Neuron Deletion

Re:Understanding Deep Learning Through Neuron Deletion by pilooch on Friday March 23, 2018 @06:51AM (#16656704)

Interesting, any public pointer to your dissertation ?

[ Parent | on HN ]

Re:Understanding Deep Learning Through Neuron Deletion by LoSboccacc on Friday March 23, 2018 @07:29AM (#16656916)

I've lost the original files, I digitalized some of the important part back, here https://imgur.com/a/w2BCy

it was 2004-2006, we used weight quantization as noise function and a modified levemberg-marquad for training, matlab was the hot stuff back then and by today standard everything is ancient, so there's that - the training alghorithm was changed to train toward a flatter area of the noise function, so it used the second derivative to hold gradient descent and overfitting.

I can answer specific questions if you want.

[ Parent | on HN ]

Re:Understanding Deep Learning Through Neuron Deletion by pilooch on Friday March 23, 2018 @10:47AM (#16658678)

Many thanks for the thorough answer! Will read and ask questions if any. We have customers we help quantize the models for their FPGAs and this may be an avenue we could test.

[ Parent | on HN ]

Re:Understanding Deep Learning Through Neuron Deletion by p1esk on Saturday March 24, 2018 @02:00PM (#16667885)

How do you currently quantize the models?

[ Parent | on HN ]