Re:Why Are Eight Bits Enough for Deep Neural Networks?
by fgimenez on Saturday September 19, 2015 @09:13PM
(#10246176)
I had similar empirical results on one of my PhD projects for medical image classification. With small data sets, we got better results on 8-bit data sets compared to 16-bit. We viewed it as a form of regularization that was extremely effective on smaller data sets with a lot of noise (x-rays in this case).
[ Parent
| on HN ]
Re:Why Are Eight Bits Enough for Deep Neural Networks?
by tachyonbeam on Sunday September 20, 2015 @03:13AM
(#10246813)
When using 8-bit weights, what kind of mapping do you do? Do you map the 8-bit range into -10 to 10? Do you have more precision near zero or is it a linear mapping?
Re:Why Are Eight Bits Enough for Deep Neural Networks?
by LoSboccacc on Sunday September 20, 2015 @05:08AM
(#10246979)
Don't know about him but I was working with -8 8 for input and -4 4 for weights, using atan function for transfer maps quite well and there is no need to oversaturate the next layer.