Re:GCC optimized code gives strange floating point results
by SolarNet on Friday March 03, 2017 @03:51PM
(#13786394)
> -sigma < x < 0 < y < sigma, abs(x - y) > sigma
This statement works fine. If the numbers are farther than sigma apart then `abs(x - y) > sigma` is true as it should be. It's not distance to 0, it's distance from each other related to 0 that matters.
> There are also problems with granularity. Suppose x and y are both much, much larger than sigma
This is part of the same problem. You have to choose a maximum acceptable value. And ensure you don't overflow it, and then you can choose a sigma based on that maximum value such that the difference between any two numbers never exceeds sigma. In practice you already have a maximum value for your domain that the simulation should never exceed or it means it is incorrect anyway.
There are also problems with granularity. Suppose x and y are both much, much larger than sigma, and the smallest representable difference between x and y is e.g. 1.0. Now there's no possible way that abs(x - y) < sigma.
I suspect that the resolution to this is to compare equality within some number of ULPs, but: 1) You still have the transitivity problem and 2) This is not my field of expertise, and there may be a more appropriate solution.