Re:Formatting floating point numbers
by Dylan16807 on Friday June 14, 2019 @09:36PM
(#20187931)
> if more users were aware of just how many of the simple rational decimals they input were converted into imprecise representations they probably wouldn't trust computers as much as they do
I disagree. People overestimate the accuracy of decimal encoding so much more than they ever overestimate floating point. Then when they see 0.1 + 0.2 they tend to learn entirely the wrong lesson, and start underestimating the accuracy of floating point alone.
> TL;DR formatting floats to decimals makes us trust floating point math far more than we should
The only reason a decimal encoding can cause too much trust is because we trust decimal too much. Decimal fails in exactly the same ways.
I disagree. People overestimate the accuracy of decimal encoding so much more than they ever overestimate floating point. Then when they see 0.1 + 0.2 they tend to learn entirely the wrong lesson, and start underestimating the accuracy of floating point alone.
> TL;DR formatting floats to decimals makes us trust floating point math far more than we should
The only reason a decimal encoding can cause too much trust is because we trust decimal too much. Decimal fails in exactly the same ways.