News for hackers. Stuff that matters.
Re:Why Are Eight Bits Enough for Deep Neural Networks? by fgimenez on Saturday September 19, 2015 @09:13PM (#10246176)
I had similar empirical results on one of my PhD projects for medical image classification. With small data sets, we got better results on 8-bit data sets compared to 16-bit. We viewed it as a form of regularization that was extremely effective on smaller data sets with a lot of noise (x-rays in this case).
Re:Why Are Eight Bits Enough for Deep Neural Networks? by tachyonbeam on Sunday September 20, 2015 @03:13AM (#10246813)
When using 8-bit weights, what kind of mapping do you do? Do you map the 8-bit range into -10 to 10? Do you have more precision near zero or is it a linear mapping?
Re:Why Are Eight Bits Enough for Deep Neural Networks? by LoSboccacc on Sunday September 20, 2015 @05:08AM (#10246979)
Don't know about him but I was working with -8 8 for input and -4 4 for weights, using atan function for transfer maps quite well and there is no need to oversaturate the next layer.