News for hackers. Stuff that matters.

A comment on FloatX: A C++ Library for Customized Floating-Point Arithmetic.

Re:FloatX: A C++ Library for Customized Floating-Point Arith... by helltone on Wednesday January 08, 2020 @09:36AM (#21991783)
I have encountered many real world cases where I needed less precision/range than what float/double had available. But usually I found fixed point was a better solution than reduced precision floats. I wonder what applications are there that can deal with reduced precision but somehow still need the range you get with an exponent?
[ Parent | on HN ]
Re:FloatX: A C++ Library for Customized Floating-Point Arith... by CodesInChaos on Wednesday January 08, 2020 @11:58AM (#21993222)
16-bit floats with an 8-bit exponent and a 7+1 bit mantissa are popular for neural networks, because they have the same range as standard 32-bit floats while taking have the memory and memory bandwidth.

https://en.wikipedia.org/wiki/Bfloat16_floating-point_format

[ Parent | on HN ]
Re:FloatX: A C++ Library for Customized Floating-Point Arith... by helltone on Wednesday January 08, 2020 @04:14PM (#21995868)
Interesting, although I believe most neural nets nowadays have moved to use linear (relu) activation, which again removes the need for exponent and would work really well with fixed point.

Here's a reference using 16-bit fixed point neural nets http://ieeexplore.ieee.org/document/7011421/?part=1

[ Parent | on HN ]