News for hackers. Stuff that matters.
Re:High Performance Correctly Rounded Math Libraries for 32-... by thxg on Thursday August 26, 2021 @04:22PM (#28320147)
Yes, indeed it is the case with fused multiply-add operations, as I wrote below in another thread. By default at -O3, unfortunately, gcc and icc both take responsibility for deciding whether or not to use FMA instructions when they see (a x b + c) expressions. As a result we can even see slightly different result across compiles!!

However, as far as I understand, specifying a proper language standard (for example -std=c99, -std=c++14, etc.) restores sane behavior (see -ffast-math, and more specifically -fexcess-precision).

Are there other cases where -O3 breaks IEEE754 on modern setups?

Re:High Performance Correctly Rounded Math Libraries for 32-... by jcranmer on Thursday August 26, 2021 @05:46PM (#28321316)
Fast math flags are actually far more complicated in C/C++ compilers than just a simple -ffast-math flag, although I suppose most users may not be acquainted with the degree of control possible. In LLVM, for example, fast math flags are actually a set of 7 independently-toggled flags (no-signed-zero, no-infinity, no-NaN, allow FMAs, allow reassociation, allow approximate reciprocal, and allow approximate functions), that at the C/C++ level can be toggled either by command-line options or on a per-scope basis. ICC actually has two different levels of fast math [1].

The dark underside of IEEE 754 is that full support for it also requires features that most languages don't provide access for: rounding mode and exception handling (aka sticky bits). In C/C++, using these features correctly requires STDC FENV_ACCESS support which is only in ICC and recent clang (neither GCC nor MSVC support it). And there's usually two hardware bits for handling denormals (flush-to-zero and denormals-are-zero) that aren't in IEEE 754, which tend to be on by default when you're compiling for some targets (most notably GPUs).

[1] https://software.intel.com/content/www/us/en/develop/documen... -- note fp-model=fast=[1|2]

Re:High Performance Correctly Rounded Math Libraries for 32-... by BeeOnRope on Saturday August 28, 2021 @12:45AM (#28335692)
What is a "per scope basis"?
Re:High Performance Correctly Rounded Math Libraries for 32-... by jcranmer on Sunday August 29, 2021 @10:38AM (#28347050)
You can change it between any pair of curly braces.

E.g.:

  void foo(int N) {
    for (int i = 0; i < N; i++) {
      #pragma STDC FENV_ACCESS ON
      // This applies only to the for loop...
    }
    // But not here, for example!
  }
Re:High Performance Correctly Rounded Math Libraries for 32-... by wumpus on Thursday August 26, 2021 @06:06PM (#28321577)
> Yes, indeed it is the case with fused multiply-add operations, as I wrote below in another thread

This is not always true. Some FMA operations round after the multiply, and some do not.