News for hackers. Stuff that matters.
Re:Beware of fast-math by zoomablemind on Friday November 12, 2021 @12:52PM (#29202345)
On the subject of the floating-point math in general, I wonder what's the practical way to treat the extreme order values (close to zero ~ 1E-200, or infinity ~ 1E200, but not zero or inf)? This can take place in some iterative methods, expansion series, or around some singularities.

How reliable is it to keep the exreme orders in expectation that the resp. quatities would cancel the orders properly yielding a meaningful value (rounding wise)?

For example, calculating some resulting value function, expressed as

v(x)=f(x)/g(x),

where both f(x) and g(x) are oscillating with a number of roots in a given interval of x.

Re:Beware of fast-math by simonbyrne on Friday November 12, 2021 @02:22PM (#29203474)
The key thing about floating point is that it maintains relative accuracy: in your case, if you have say f(x) and g(x) are both O(1e200), and are correct to some small relative tolerance, say 1e-10 (that is, the absolute error is 1e190). Then the relative for f(x)/g(x) stays nicely bounded to about 2e-10.

However if you do f(x) - g(x), the absolute error is on the order of 2e190: if f(x) - g(x) is small, then now the relative error can be huge (this is known as catastrophic cancellation).

Re:Beware of fast-math by zoomablemind on Friday November 12, 2021 @02:54PM (#29203816)
Both f(x) and g(x) could be calc'ed to proper machine precision (say, GSL double prec ~ 1E-15). Would this imply, that beyond the machine precision the values are effectively fixed to resp. zero or infinity, instead of carrying around the extreme orders of magnitude?
Re:Beware of fast-math by simonbyrne on Friday November 12, 2021 @03:21PM (#29204079)
I'm not exactly sure what you're asking here, but the point is that "to machine precision" is relative: if f(x) and g(x) are O(1e200), then the absolute error of each is still O(1e185). However f(x)/g(x) will still be very accurate (with absolute error O(1e-15)).
Re:Beware of fast-math by gsteinb88 on Friday November 12, 2021 @04:37PM (#29204834)
If you can, working with the logarithms of the intermediate large or small values is one way around the issue

One example talking about this here: http://aosabook.org/en/500L/a-rejection-sampler.html#the-mul...

Re:Beware of fast-math by junon on Friday November 12, 2021 @12:58PM (#29202417)
Not very reliable. For precision one usually seeks out the MPFR library or something similar.
Re:Beware of fast-math by zoomablemind on Friday November 12, 2021 @01:13PM (#29202623)
If anyone else wonders, MPFR (Multi-Precision Floating-point calc with correct Rounding). The application domain is Interval Math [1].

[1]:http://cs.utep.edu/interval-comp/applications.html

Re:Beware of fast-math by adgjlsfhk1 on Friday November 12, 2021 @01:20PM (#29202730)
ARB https://arblib.org/ is often better than mpfr.
Re:Beware of fast-math by junon on Friday November 12, 2021 @08:47PM (#29206570)
Thanks, that's a new one.