Re:Floating point numbers, and why they suck
by kstenerud on Friday December 17, 2021 @12:58PM
(#29595731)
I think I need to do a blog post on this. I get the same kind of reaction from pretty much anyone I talk to about this, and it takes hours of explanations & diagrams to get them to understand. It's a real shame too, because a correct understanding would have a LOT more people putting pressure on implementors to support decimal floats :/
The article itself actually started with your assumption... and found it wasn't true. The type conversion wasn't the issue--it was coming back as the same value on all the different systems.
The flaw was that the sum came out wrong depending on the order that it was done. And the input numbers were (if I'm understanding correctly) computed as a / b, for some integer values a and b, which are not generally perfectly precise in decimal or binary floating point.