Re:Towards 1-bit Machine Learning Models
by overengineer on Saturday March 30, 2024 @06:53AM
(#39873627)
If noise is desirable in this way, why not to just switch back to analog computing which comes with free noise
I was able to get 0.68 "effective" bits.
The idea is that in each forward pass you add noise to each weight independently drawn from normal distribution, and when you calculate snr, it's sub 1 bit. Points to the idea that a stochastic memory element can be used.
https://arxiv.org/abs/1606.01981