News for hackers. Stuff that matters.
Re:The Fundamental Axiom of Floating Point Arithmetic by tux3 on Wednesday August 05, 2020 @03:16AM (#24058086)
I'm far from a floating point expert, but the fact that the article does everything in terms of epsilons, without mentioning ULPs a single time confuses me.

In fact, that fundamental axiom of FP arithmetic is NOT in terms of some absolute value epsilonMachine. As I understand it, it says that there exists a relative epsilon value (which depends on your input numbers), and that the error of any FP operation on those numbers will be bounded by that epsilon.

What the article doesn't explain, is that floating point numbers and operations are guaranteed to be accurate to the nearest half-ULP, not to the nearest absolute epsilon. And what exactly is the absolute value of an ULP? It depends!

The difference is so fundamental, I'm not sure you can have a useful mental model of floats if you don't start by addressing this.

Re:The Fundamental Axiom of Floating Point Arithmetic by akdor1154 on Wednesday August 05, 2020 @05:11AM (#24058721)
The epsilon model is perfectly adequate, and does not assume epsilon is constant on a given machine (the article glosses over this in the first paragraph where epsilon is introduced, but it is spelled out later).

In fact the epsilon model is key to being able to do a mathematically formulated error analysis at all.

Re:The Fundamental Axiom of Floating Point Arithmetic by kd5bjo on Wednesday August 05, 2020 @04:10AM (#24058378)
Would you mind expanding the `ULP` acronym you’re using: if it’s so fundamental to understanding this, I’d like to know what it is.
Re:The Fundamental Axiom of Floating Point Arithmetic by tux3 on Wednesday August 05, 2020 @04:15AM (#24058407)
My apology, ULP stands for "Unit in the Last Place".

The simple explanation is that an ULP is defined as the difference between two consecutive floating point numbers, which means it isn't a fixed value.

Another "intuitive" explanation, is that if you increment the binary representation of a float, the difference between the two numbers is 1 ULP.

(I'd explain more, but I have to run to a meeting! More info on Wikipedia: https://en.wikipedia.org/wiki/Unit_in_the_last_place)

Re:The Fundamental Axiom of Floating Point Arithmetic by DougBTX on Wednesday August 05, 2020 @07:19AM (#24059489)
Instead of calling it "ULPs" they refer to a "next floating point number" which I'm pretty sure is the same idea:

> Unlike the number line, for every representable floating point number x, there is a _next_ floating point number, and it is approximately x + ϵmachine x. So for an arbitrary real number x0, it falls between two floating point values x and x+ϵx (leaving off the subscript for conciseness). When we represent x0 in floating point, we will get one of these two values.

Re:The Fundamental Axiom of Floating Point Arithmetic by fractionalhare on Wednesday August 05, 2020 @08:55AM (#24060264)
Machine epsilon is fine for doing error analysis. For example, I used machine epsilon instead of ULP for an article I recently wrote which walks through implementation and (numerical) analysis of various algorithms for e^x: https://www.pseudorandom.com/implementing-exp

So you don't need ULP (and if you did, you could convert to ULP from machine epsilon). But you are right that machine epsilon varies according to where you are in the interval of floating point numbers. Some authors don't mention that, but yes, since floating point values are not evenly distributed the machine epsilon for any particular x will vary. This is good to know, because even if error analysis demonstrates that your approximation function is bounded by machine epsilon, it's still going to exhibit less accuracy as you move away from the origin.

Re:The Fundamental Axiom of Floating Point Arithmetic by carlmr on Wednesday August 05, 2020 @04:33AM (#24058521)
But multiplying by (1+e) is relative. I guess it's much more convenient mathematically than dealing with the absolute error of the calculation.