News for hackers. Stuff that matters.
Re:0.30000000000000004 by fryguy on Friday November 13, 2015 @01:17PM (#10561356)
In decimal if you only have 5 digits for mantissa and you write 2/3, it becomes 0.66667 which is a little bit more than 2/3. Similarly, if you write the binary number 0.0000'0000'1 in decimal with 5 digits for mantissa, it becomes 0.0039063 instead of 0.00390625. If in some faraway land they use binary math, they would likely ask why such a simple binary number got rounded off.

There are no comments on this comment yet.