Re:What Every Computer Scientist Should Know About Floating-...
by hnick on Monday June 29, 2020 @04:55AM
(#23675728)
Yes, that is true. But usually symptoms don't surface until errors accumulate which is why I found that web form interesting.
'Precise' was the wrong word. What I meant was the system reports back the exact entered value 0.1 (rather than what it has in memory, which is 0.100000001490116119384765625 according to https://www.h-schmidt.net/FloatConverter/IEEE754.html). Most programs will do this fairly well.
I confirmed javascript is entirely capable of storing and retrieving the number I entered, so I'm not entirely sure what they were doing to confuse it :) Must have been some secret maths.
'Precise' was the wrong word. What I meant was the system reports back the exact entered value 0.1 (rather than what it has in memory, which is 0.100000001490116119384765625 according to https://www.h-schmidt.net/FloatConverter/IEEE754.html). Most programs will do this fairly well.
I confirmed javascript is entirely capable of storing and retrieving the number I entered, so I'm not entirely sure what they were doing to confuse it :) Must have been some secret maths.