Re:9999999999999999.0 – 9999999999999998.0
by jimmaswell on Saturday January 05, 2019 @07:36PM
(#18835453)
My CS course at a community college went over the limitations of floats in detail, in the CS class.
[ Parent
| on HN ]
Re:9999999999999999.0 – 9999999999999998.0
by copperx on Saturday January 05, 2019 @11:25PM
(#18836562)
How detailed, if I may ask?
Re:9999999999999999.0 – 9999999999999998.0
by jimmaswell on Monday January 07, 2019 @02:38PM
(#18849130)
I remember we learned how the representation worked bit-for-bit, and how it being stored in binary meant it couldn't perfectly represent everything in decimal. 1.01b meaning 12^0 + 02^-1 + 1*2^-2 for example.