News for hackers. Stuff that matters.
Re:GCC optimized code gives strange floating point results by castratikron on Sunday February 26, 2017 @02:35PM (#13739283)
More proof that floats are evil. Moral of the story is to never use equality to compare floats.
Re:GCC optimized code gives strange floating point results by fdej on Sunday February 26, 2017 @03:52PM (#13739785)
That's not the issue here at all. The IEEE 754 standard does in principle make floating-point operations completely deterministic, i.e. if you do a specific sequence of floating-point operations, then the result may indeed be slightly off from the result you would expect with infinite precision arithmetic, but it should be off in just the same way every time. For example (x + 0.1) + 0.1 and x + 0.2 can give different results due to rounding, but the respective result should be constant for a given x regardless of the machine, compiler, or phase of the moon. Knowing this, there are situations where exact comparisons are appropriate.

The "bug" here is that a single operation x + 0.1 and x + 0.1 can give two different results depending on the mood of the compiler.

It's not actually a bug, because the IEEE 754 standard permits this kind of sloppiness by default. You can get stricter semantics, but you have to ask for it explicitly by setting the right compiler flags. This is why I say that the IEEE 754 standard makes floating-point arithmetic deterministic "in principle" - in practice, you have to work for it. (Changes to the rounding mode are another big issue here.)

An extra error of 1 ulp somewhere might not seem like much for your average numerical code, but it's a big deal in some situations:

* Assumptions about the precise behavior of floating-point cancellations are often used in numerical libraries, and deviations can cause disastrous loss of accuracy. For example, Kahan summation is a widely used algorithm for computing sums with increased accuracy. If compilers optimize the code too aggressively, the algorithm doesn't work and you get worse accuracy. And ironically, x87 long doubles can make general numerical calculations less accurate (not more accurate as intended) due to the double rounding issue.

* If you use floats in game logic, you might want it to behave exactly the same way on different platforms, e.g. for synchronization or replays.

* In scientific computing, you might want results to be precisely reproducible by others.

* Debugging!

Re:GCC optimized code gives strange floating point results by tbirdz on Sunday February 26, 2017 @05:37PM (#13740429)
In practice, I don't think IEEE 754 floating point is deterministic, especially not across multiple ISAs, multiple vendors of the same ISA (Intel, AMD), different generations of the same vendor's cpus, different compilers and different math libraries, even just C's libm.

So just a warning to everyone out there: unless you are very, very careful your floating point code will not be deterministic. If you thought you can just use floats and expect it to "just work" the same, then you're out of luck.

Here's a couple of articles which go into more detail about problems with floating point determinism:

http://gafferongames.com/networking-for-game-programmers/flo...

https://randomascii.wordpress.com/2013/07/16/floating-point-...

Re:GCC optimized code gives strange floating point results by gpderetta on Monday February 27, 2017 @08:41AM (#13744034)
As long as the unit is IEE754, results are expected to be exact to the last ulp [1]. There might be issues in the lowering of high level languages to low level assemblers, but that's a different story (i.e the implementation is not IEE754 compliant).

[1] transcendental functions IIRC are not required to be exact to the full precision and historically they aren't. But there are (portable) math libraries that guarantee full precision.

Re:GCC optimized code gives strange floating point results by trendia on Sunday February 26, 2017 @02:48PM (#13739375)
Pretty much. If you want to compare two floats a and b within some precision eps, use:

    abs(a - b) <= eps*0.5*(a + b)
or even

    (abs(a - b) <= eps*abs(a)) && (abs(a - b) <= eps*abs(b))
which is implemented in Boost [0].

[0] http://www.boost.org/doc/libs/1_34_0/libs/test/doc/component...

Re:GCC optimized code gives strange floating point results by ratboy666 on Sunday February 26, 2017 @03:33PM (#13739666)
I thought it was

(a / b) != (a / b)

Re:GCC optimized code gives strange floating point results by nom on Sunday February 26, 2017 @03:03PM (#13739464)
Floats are not evil. They are just not as intuitive as integers because it is impossible to represent every possible rational number in the range with a fixed number of bits. Once you get that, everything makes sense.
Re:GCC optimized code gives strange floating point results by monochromatic on Sunday February 26, 2017 @04:07PM (#13739868)
That does not make sense of getting a failure from

assert x/y == x/y

Re:GCC optimized code gives strange floating point results by cellularmitosis on Sunday February 26, 2017 @05:07PM (#13740249)
I agree. Explaining that assertion requires explaining that the x87 FPU is providing a leaky abstraction. If you ask for double math, it gives you a result which is a long double while it resides in a register, and then becomes a double when moved to RAM.

There are some cases where direct comparison of floating point values is what you want, and this issue can still bite you even then.

Re:GCC optimized code gives strange floating point results by int_19h on Sunday February 26, 2017 @05:21PM (#13740345)
For x87, it's not a leaky abstraction - it fully supports 80-bit floats as a type, and documents that its registers are 80-bit, and so are operations on them. There's nothing abstract about it.

The bug is that GCC is utilizing the (documented, by-design) x87 behavior in a way that causes it to produce the wrong result, for the sake of performance.

Re:GCC optimized code gives strange floating point results by abstractbeliefs on Sunday February 26, 2017 @06:16PM (#13740586)
>compiler developers do unintuitive but technically legal thing in the name of performance

I still think the relationship between compiler developers and developer-users is toxic as fuck, but as long as we keep comparing compilers by increasingly anal microbenchmarks instead of increasingly anal "correct in practice" tests, I don't see why we continue to find this a surprise.

When we demand better performance, compiler developers are going to exploit undefined behavior as well as other language "loopholes" that don't make intuitive sense. Perhaps a new focus on correctness in practice would change this.

Re:GCC optimized code gives strange floating point results by mikeash on Sunday February 26, 2017 @05:04PM (#13740233)
Properly implemented floats will never fail that (barring NaN). That gcc sometimes does is a compiler bug, not a float problem.
Re:GCC optimized code gives strange floating point results by geezerjay on Sunday February 26, 2017 @02:51PM (#13739392)
> More proof that floats are evil.

Just because someone doesn't know the basics of floating point numbers it doesn't mean that the floating point format is somehow evil.

Testing floats for equality is one of those no-nos which is taught right in the first class of introduction to FP numbers.

Another very basic issue covered right in the intro clsses are problems caused by using decimal values to define numbers encoded in binary format, which is often explained through the famous "0.1 doesn't exist in binary" trivia bit.

Re:GCC optimized code gives strange floating point results by thechao on Sunday February 26, 2017 @03:25PM (#13739623)
Part of my day job is to test floating point hardware and write compilers and assemblers. Testing equality---with some care---is completely fine. I can assure you this is a bug in GCC. The GCC bug is compounded by the nonstandard behavior of x87. The x87 unit is not "fixable" but GCC's behavior is.
Re:GCC optimized code gives strange floating point results by nurettin on Monday February 27, 2017 @12:05AM (#13742019)
That's interesting, you make it sound like the production of new FPUs and new compilers is common. Which industry do you work in?
Re:GCC optimized code gives strange floating point results by yongjik on Sunday February 26, 2017 @03:49PM (#13739771)
There are numbers that are safe to compare against. Storing 1.0 (or 0.0 or 0.25 or any small integer), and then later comparing for equality to check "Did I assign this value before?" is completely fine.

Of course, it's not fine to ask "Does the result of this computation equal this value?", but everyone knows that.

Re:GCC optimized code gives strange floating point results by jobigoud on Monday February 27, 2017 @06:23AM (#13743434)
> Of course, it's not fine to ask "Does the result of this computation equal this value?", but everyone knows that.

How about "Does the result of this computation equal the result of this exact same computation?"? That's the issue here.

Re:GCC optimized code gives strange floating point results by andrepd on Sunday February 26, 2017 @04:23PM (#13739968)
Floats are evil? o.O What do you propose instead if you want to deal with non-integer values?
Re:GCC optimized code gives strange floating point results by tbirdz on Sunday February 26, 2017 @05:25PM (#13740370)
There's always fixed point integers[0]. They're less flexible than floats, but at least the amount of precision available is much more clear and uniform, and all the rounding is pretty explicit.

https://en.wikipedia.org/wiki/Fixed-point_arithmetic

https://en.wikipedia.org/wiki/Q_%28number_format%29

Re:GCC optimized code gives strange floating point results by rwallace on Sunday February 26, 2017 @08:32PM (#13741210)
Back in the late eighties, early nineties, I was of the opinion that fixed point is really the way to go, the only thing holding it back was lack of adequate compiler support. Until I realized what would actually happen if the world started using fixed point: programmers would always set the dynamic range too small, so we'd end up with a world full of programs that were constantly failing because they needed to be run on data larger than the test cases. The headaches of floating point, annoying though they can be at times, are much less onerous.
Re:GCC optimized code gives strange floating point results by MaulingMonkey on Monday February 27, 2017 @12:44AM (#13742150)
> Until I realized what would actually happen if the world started using fixed point: programmers would always set the dynamic range too small, so we'd end up with a world full of programs that were constantly failing because they needed to be run on data larger than the test cases.

Floating point doesn't actually fix this issue. A fun bug I dealt with was a black screen caused by a few pixels having NaN values - which were then picked up by a blur pass which would poison basically the entire back buffer. The original NaNs were caused by divides by near-zero values causing out-of-range results.

You now add precision issues: Multiple coordinate systems are a common hack-around to maintain local precision when using single-precision based physics engines. Stuttering effects when time is represented as "floating point seconds since uptime" when integer wraparound would avoid any problem in the first place. And this is before even adding e.g. half precision or worse floats into the mix.

There's a reason floating point isn't used for a lot of serious banking calculations.

Re:GCC optimized code gives strange floating point results by rwallace on Monday February 27, 2017 @02:17PM (#13747000)
To clarify, any time I in any way praise floating point, I'm always talking about double precision. I think single precision floating point should never have been invented, and given that it unfortunately was, it's best to behave as though it doesn't exist. Don't get me started about half precision.

Financial calculations are another matter; they usually require decimal arithmetic, not because it doesn't give rounding errors, but because it gives the same rounding errors an accountant gets when he checks the result with his desk calculator, and that's the requirement.

Re:GCC optimized code gives strange floating point results by MaulingMonkey on Monday February 27, 2017 @06:41PM (#13749686)
> To clarify, any time I in any way praise floating point, I'm always talking about double precision.

64-bit fixed point solves a lot of problems too :)

> I think single precision floating point should never have been invented, and given that it unfortunately was, it's best to behave as though it doesn't exist. Don't get me started about half precision.

A 4x memory bandwidth & storage penalty for HDR buffers would be absolute murder (bloating a single 4k RGBA surface to 265MB if I've done my math right), so I'm glad we have the option. And in the historical context lower precision options were invented in, RAM wasn't nearly as cheap as it is today. Tradeoffs.

Re:GCC optimized code gives strange floating point results by rnhmjoj on Sunday February 26, 2017 @05:10PM (#13740284)
Continued fractions are perfect if you have infinite memory.
Re:GCC optimized code gives strange floating point results by kccqzy on Monday February 27, 2017 @02:28AM (#13742514)
A real number can contain an infinite amount of information.

If you have infinite memory, you would probably start by asking whether you have countably infinite or uncountably infinite memory.

Re:GCC optimized code gives strange floating point results by pkaye on Sunday February 26, 2017 @03:24PM (#13739617)
What would you suggest as an alternative to float?
Re:GCC optimized code gives strange floating point results by CalChris on Sunday February 26, 2017 @03:47PM (#13739759)
Re:GCC optimized code gives strange floating point results by dgudkov on Monday February 27, 2017 @11:47PM (#13751271)
128-bit decimal.