News for hackers. Stuff that matters.
Re:Why Are Eight Bits Enough for Deep Neural Networks? by srean on Sunday September 20, 2015 @01:26AM (#10246652)
I am pretty sure that parent meant DNNs love to fit noise. So you two are in strong agreement.
Re:Why Are Eight Bits Enough for Deep Neural Networks? by p1esk on Sunday September 20, 2015 @02:13AM (#10246719)
No, we are talking about a random electrical circuit noise in the analog NN hardware. Of course, if the noise is known and fixed, the net could learn to compensate (to a certain extent). The noise we are talking about is like when you put your finger on the chip, and raise its temperature by 10 degrees, the whole thing needs to be retrained.
Re:Why Are Eight Bits Enough for Deep Neural Networks? by taliesinb on Sunday September 20, 2015 @03:32AM (#10246841)
> The noise we are talking about is like when you put your finger on the chip, and raise its temperature by 10 degrees, the whole thing needs to be retrained.

What would change with temperature that would require retraining? Are you saying the output of an op could depend sensitively on temperature, or that higher temperatures would increase things like thermal or shot noise? Why would the latter require retraining?

Re:Why Are Eight Bits Enough for Deep Neural Networks? by p1esk on Sunday September 20, 2015 @12:58PM (#10248172)
Current depends on threshold voltage and carrier mobility, which are temperature dependent.