Re:Beware of fast-math
by simonbyrne on Friday November 12, 2021 @02:22PM
(#29203474)
The key thing about floating point is that it maintains relative accuracy: in your case, if you have say f(x) and g(x) are both O(1e200), and are correct to some small relative tolerance, say 1e-10 (that is, the absolute error is 1e190). Then the relative for f(x)/g(x) stays nicely bounded to about 2e-10.
However if you do f(x) - g(x), the absolute error is on the order of 2e190: if f(x) - g(x) is small, then now the relative error can be huge (this is known as catastrophic cancellation).
Re:Beware of fast-math
by zoomablemind on Friday November 12, 2021 @02:54PM
(#29203816)
Both f(x) and g(x) could be calc'ed to proper machine precision (say, GSL double prec ~ 1E-15). Would this imply, that beyond the machine precision the values are effectively fixed to resp. zero or infinity, instead of carrying around the extreme orders of magnitude?
Re:Beware of fast-math
by simonbyrne on Friday November 12, 2021 @03:21PM
(#29204079)
I'm not exactly sure what you're asking here, but the point is that "to machine precision" is relative: if f(x) and g(x) are O(1e200), then the absolute error of each is still O(1e185). However f(x)/g(x) will still be very accurate (with absolute error O(1e-15)).
If you can, working with the logarithms of the intermediate large or small values is one way around the issue
One example talking about this here: http://aosabook.org/en/500L/a-rejection-sampler.html#the-mul...
Not very reliable. For precision one usually seeks out the MPFR library or something similar.
Re:Beware of fast-math
by zoomablemind on Friday November 12, 2021 @01:13PM
(#29202623)
If anyone else wonders, MPFR (Multi-Precision Floating-point calc with correct Rounding). The application domain is Interval Math [1].
Re:Beware of fast-math
by adgjlsfhk1 on Friday November 12, 2021 @01:20PM
(#29202730)
ARB https://arblib.org/ is often better than mpfr.
How reliable is it to keep the exreme orders in expectation that the resp. quatities would cancel the orders properly yielding a meaningful value (rounding wise)?
For example, calculating some resulting value function, expressed as
v(x)=f(x)/g(x),
where both f(x) and g(x) are oscillating with a number of roots in a given interval of x.