News for hackers. Stuff that matters.
Re:GCC optimized code gives strange floating point results by hzhou321 on Sunday February 26, 2017 @07:00PM (#13740809)
People need understand that floating point comparison at floating precision is undefined, so it is not a bug. The proper floating number representation should track its precision. Since that is more complicated and potentially performance affecting, so until then the programmer should manage the precision tracking on their own (similar to memory allocations), that is, one should do:

  if ( f_a < f_b-f_eps)
  if ( f_a > f_b+f_eps)
  if (fabs(f_a-f_b)<f_eps)
If you don't do this, then you are on your own. Sometimes it is fine, but sometimes it may give you grief, just like the dangers of other undefined behaviors.

PS: I also have a problem that people think "undefined behavior" is undefined because certain standards say so. Undefined behavior is due to lack of agreement of common sense or defies common sense. For example, divide by zero, there is no common sense doing so, and define its behavior does not change anything. Signed integer overflow, there is no common sense agreement for that behavior either. The common sense for undefined behaviors is to avoid it. (On the other hand, there seems to be some common sense agreement on unsigned integer overflow, and people are writing code based on that behavior. But that agreement is weak, so I would avoid it if I can regardless what the standards say.)

PS2: The reason that floating point comparison is undefined behavior because, unlike fixed precision or integers, floating point number has floating precision. Floating precision does not have common sense correspondence. In any real application (where common sense is based), a finite, and above all, consistent precision, is always assumed.

And the solution, by manually specifying a f_eps, we put that comparison into a common-sense understanding that you know the result of comparisons is only meaningful above f_eps. With that common sense, having an internal calculation in higher precision is always OK -- isn't that common sense?

PS3: The question is why not turn undefined behaviors into errors? Undefined behavior does not mean there are no behaviors, and the behaviors can be used, or more often, ignored to achieve certain efficiency. When programmers write code with undefined behaviors, they are extracting their code out of common sense domain, but into a special sense domain. Special sense domain is practically fine. Everyone has their own efficient quirks, just do not expect your quirks to be understood or supported by out large where common sense rules (unless you carefully define it and document it). A language that allows for undefined behaviors allows for individual efficiency, which sometimes it is rather good.

For example, it is ok to write programs with direct floating point number comparisons as long as you understand that your program does not care when the comparison falls into the ambiguous range. Only you who understand your special application can assess that. If you are the only one who is responsible for the result of that code, then it is perfectly fine!

Re:GCC optimized code gives strange floating point results by enriquto on Monday February 27, 2017 @03:34AM (#13742792)
Essentially everything you say here is false.

Floating point numbers represent rational numbers, and their arithmetic is straightfoward and follows common sense: you compute the exact operation using rational numbers, and the result is the closest floating point number available.

Floating point operations are never "undefined" either, they are perfectly deterministic and completely specified by the IEEE 754 standards.

There are three floating point values that do not correspond to rationals: INF, -INF and NAN. There is also "-0" that corresponds to the same rational as "0". The arithmetic of these values is extremely common sense to anyone that knows some calculus: 1/0 = INF, INF*0=NAN, INF+INF=INF, INF-INF=NAN, etc.

There is no ambiguity whatsoever in floating point numbers.

Re:GCC optimized code gives strange floating point results by jlgustafson on Monday February 27, 2017 @05:33AM (#13743252)
You say 1/0 = INF, but the 754 standard says negative zero is different, and 1/(-0) = -INF. Yet it says -0 is numerically equal to 0. It also specifies that the square root of -0 is -0. This is neither straightforward nor common sense.

They also do not round to the nearest representable rational number. If you double MAXREAL, it will round to INF, even though MAXREAL is infinitely closer to 2*MAXREAL than INF is.

Despite efforts in the 2008 update to the 754 Standard, there still is no guarantee that the Standard will produce reproducible, bitwise identical results. The Standards committee admits this. It is important for everyone to understand the shortcomings of the Standard.

Re:GCC optimized code gives strange floating point results by enriquto on Monday February 27, 2017 @07:09AM (#13743630)
Well, "common sense" is not the same for everybody then. For a mathematician, the rules 1/0 = INF, 1/(-0) = -INF are perfectly straightforward and common sense (if you think about the limits).

Regarding sqrt(-0)=-0, I concede it looks a bit strange. Yet, as explained in the famous Kahan article "much ado about nothing's sign bit", it turns out to be natural when you work with complex numbers.

As for the nondeterminacy, I suppose you are talking about transcendental functions? The basic arithmetic should be bitwise reproductible.

Re:GCC optimized code gives strange floating point results by hzhou321 on Monday February 27, 2017 @08:59AM (#13744161)
> Essentially everything you say here is false.

That is OK, but I hope you still can get my main point whether it is true or false. If all you see is everything, I am not sure you understand what I was saying. You have to understand what I was saying in order to argue back -- unless all you care is about right or wrong rather than making progress on understanding.

My main point is to point out why people keep filing these bug reports on floating point results that compilers simply will not fix. That is because, in real applications, we expect numbers to have persistent precisions. Persistent precisions make sense in real world. For example, if you take a measurement using one ruler in mm, that precision does not change whether you are measuring rice or elephant. When you are dealing with money, one always expect the precision do not go beyond cent regardless of what your internal calculation says -- (5.001 dollars is equal to 5 dollars). And in real life calculating with higher precision than its requirement will never result in error. My career is in scientific computing and we routinely deal with very small (or very large numbers). However, in a certain application, a persistent precision is always expected, and numbers within that precision become ambiguous in comparison. For example, if my precision is around 0.001, a holds 5, b hold 5.0008, is a<b? It depends. And if your critical application -- e.g. a feedback control loop -- depends on what is defined by floating point operation, you application can be very unpredictable.

I hate to repeat but your reply didn't address my main point at all -- fixed precision expectation in reality vs. arbitrary precision in computer and vs. infinite precisions in math. Common sense is rooted to reality.

Yes, I understand floating point operation by computer under a fixed width is not really undefined. That is why I made extra points commenting on what do we mean by undefined in a computer language design. Any computer architecture with any given compiler implementation will always give definite results for any input. Undefined behavior has a root meaning from language designer point of view. For example, division, we understand what we intend for how division work, whether it is in integer case or floating point case. But do we understand what we want when the divisor is 0 or near 0? We understand we don't want the computer always crash so we don't specify that is an error, but we don't know what it should do. So we specify that this is an undefined behavior. I understand that there is INF that certain mathematicians perfer. But are you sure that is the common sense behavior? For some of my application, I would rather it is a valid big number, and for others, I would rather it is an error. In stead of wish, I learn to put checks around critical area. And, please hear this, in no application that would like an INF result, INF or -INF. There is no common sense interpretation for INF, other than treat it same as error.

So do understand what ambiguity means. In math, you never have ambiguity. In computer, you never have ambiguity. In real life common sense, ambiguity arises when we can't decide a favorable general solution.