Re:0.30000000000000004
by fryguy on Friday November 13, 2015 @01:17PM
(#10561356)
In decimal if you only have 5 digits for mantissa and you write 2/3, it becomes 0.66667 which is a little bit more than 2/3. Similarly, if you write the binary number 0.0000'0000'1 in decimal with 5 digits for mantissa, it becomes 0.0039063 instead of 0.00390625. If in some faraway land they use binary math, they would likely ask why such a simple binary number got rounded off.
[ Parent
| on HN ]
There are no comments on this comment yet.