Re:XNOR-Net: ImageNet Classification Using Binary Convolutio...
by Houshalter on Sunday March 20, 2016 @10:52PM
(#11325901)
Logarithms are both expensive to compute, and don't magically add any more bits of accuracy.
[ Parent
| on HN ]
Re:XNOR-Net: ImageNet Classification Using Binary Convolutio...
by kuschku on Monday March 21, 2016 @04:52AM
(#11326865)
Well, (a) 2-based logarithms are cheap to compute, and (b) add quite some bits of accuracy if the alternative is rounding the value to 0.
Re:XNOR-Net: ImageNet Classification Using Binary Convolutio...
by Houshalter on Tuesday March 22, 2016 @01:31AM
(#11334603)
They are not cheap to compute, I believe the best way to approximate them is using expensive polynomial approximation which consumes a few cycles and operations.
All they do is map the numbers into a different range. A logarithmic value can represent more numbers near 0, but in order to add two logarithms, they need to be converted back to normal form, where the small numbers are rounded down to zero again.
There's no way around that, as adding a very small number to a very large number will always require many bits of precision to do accurately, regardless what transforms you use on the numbers.