News for hackers. Stuff that matters.
Re:C for All by geezerjay on Friday March 23, 2018 @03:28PM (#16661787)
> Improvements to C would be things like removing "undefined behavior"

This nonsense again. I don't get this "undefined behavior" cliche. It seems it became fashionable for some people to parrot it like a mantra as a form of signaling. Undefined behavior just refers to something that is not covered by the international standard, and therefore doesn't exist nor should be used, but an implementation may offer implementation-specific behavior.

Re:C for All by derefr on Friday March 23, 2018 @03:46PM (#16661960)
When people talk about "removing undefined behavior", they usually mean requiring that compile-time-detectible undefined behaviors be converted into explicit errors.

For example, there are quite a few people who would like to see a C where you can't actually write this:

    // int x, y;
    if(++x < y) { ... }
...because, well, the behavior of integer overflow is undefined in C, so that code could technically do anything, even though it seems perfectly innocent, especially when coming from a checked language.

Of course, you can't do anything in the C standard to require that this code work as-is, because the C standard applies to architectures where mutually-exclusive things happen under integer overflow. But you can always just disallow it completely, and require that people use intrinsics that are explicit about what overflow behavior they expect (where that behavior reduces to plain output on target architectures that follow it, and to a shim on target architectures that don't. You know, like floating-point support, or atomics.)

Re:C for All by userbinator on Friday March 23, 2018 @09:53PM (#16664303)
...because, well, the behavior of integer overflow is undefined in C, so that code could technically do anything

No compiler is going to go out of its way to compile an increment into the machine's usual increment instruction and an additional overflow check that does whatever, just because it can. It's going to compile it into the machine's usual increment instruction and what happens on overflow is what happens naturally.

It's as absurd as claiming that even "x + y" can invoke undefined behaviour, because while the standard allows it, any compiler that compiles such an addition to anything other than the machine's addition instruction (i.e. with implementation-defined effects), to speak nothing of adding the additional(!) checks to deliberately do something else, is clearly not benefiting anyone.

"In theory, there is no difference between theory and practice. In practice, there is." Pure fearmongering, IMHO.

Re:C for All by MaulingMonkey on Saturday March 24, 2018 @12:50AM (#16664935)
> No compiler is going to go out of its way to compile an increment into the machine's usual increment instruction and an additional overflow check that does whatever, just because it can

What people are actually worried about is when the compiler starts removing - not adding - seemingly unrelated code in a hard to reason about fashion. And compilers absolutely will go out of their way to do this in the name of optimization and performance. And it will do this because it got smart enough to prove that the "unrelated" code can't run without first technically invoking undefined behavior, at which point it can jump to the wild conclusion that it must never actually execute (or that it can remove the code even if it does, because it's legal for the compiler to do anything after invoking undefined behavior - including not execute that code!)

Sometimes the removed code is important security checks, leading to CVEs, hotpatches, etc. - this is not theoretical, and is not remotely new at this point: https://www.grsecurity.net/~spender/exploits/cheddar_bay/exp...

It also makes reporting compiler bugs annoying, as you first have to definitively prove to yourself and the compiler guys that you've actually got a compiler bug, rather than a compiler "feature" of aggressive optimization within the letter of the C++ standard. It's only out of pure stubbornness that https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84658 got reported upstream, I was assuming it was UB in our codebase most of the way down and thus INVALID as a compiler bug...

But perhaps CVEs and expected behavior being borderline indistinguishable from compiler bugs to most C and C++ programmers I know is just "fear mongering" as you say. IMNSHO, it's not \o/

Re:C for All by teraflop on Friday March 23, 2018 @10:47PM (#16664494)
It's not about the compiler doing "extra checks" to deliberately do something different. The real issue is with aggressive optimizations.

In order to get maximum performance, the compiler is allowed to assume that the programmer doesn't invoke undefined behavior. In other words, it can replace code with something that is equivalent in the presence of UB, but does something totally different in the absence of UB. See e.g. https://blog.regehr.org/archives/767 for some examples of how this can go wrong. (My favorite is the third one.)

Re:C for All by jhomedall on Saturday March 24, 2018 @10:25AM (#16666733)
Modern C compilers do some very tricky things with undefined behavior.

See https://blogs.msdn.microsoft.com/oldnewthing/20140627-00/?p=... for a very detailed example.

See https://blog.regehr.org/archives/1307 for some strict aliasing examples.

Re:C for All by roywiggins on Friday March 23, 2018 @10:56PM (#16664528)
Compilers deciding to elide code paths that contain undefined behavior is weird, especially when it chooses to silently elide your checks for division by zero or overflow. It's not weird that actually dividing by zero can do anything; it's weird that by having a possible division by zero can allow the compiler to decide that (divisor==0) is false and ignore it.
Re:C for All by geezerjay on Friday March 23, 2018 @03:56PM (#16662069)
> they usually mean requiring that compile-time-detectible undefined behaviors

I don't believe that's the case for plenty of reasons, such as:

- compilers already do that ( yeah, it's one of those RTFM things. See for example GCC's undefined behavior sanitizer)

- the standard already specifies exactly what it is left undefined, thus it's a compiler-related issue (see point above)

Let's face it: some people mindlessly parrot the"undefined behaviour" mantra just for show.

Re:C for All by derefr on Friday March 23, 2018 @04:00PM (#16662115)
> compilers already do that ( yeah, it's one of those RTFM things)

I believe 99% of what people care about the C standard doing, re: UB handling, is requiring compilers to make certain behaviours the default, rather than hidden behind different flags that C newbies who don't understand UB (who thus code most of the bad C!) won't ever set.

Re:C for All by jcelerier on Friday March 23, 2018 @04:51PM (#16662543)
> - compilers already do that ( yeah, it's one of those RTFM things. See for example GCC's undefined behavior sanitizer)

well, no they don't. UBSan is at run-time because most UB is impossible to catch at compile-time.

Re:C for All by knome on Friday March 23, 2018 @03:42PM (#16661927)
Implementation defined behavior is not the same thing as undefined behavior.

Undefined is out of the scope of the language entirely. Using a non-existent index into an array, for example. While you might reasonably expect the program will just look past the end, there is not guarantee it will do so. Optimizing compilers, in particular will assume such a thing cannot happen, and can assume a code branch that does something like this is impossible to reach and discard it entirely.

No one will "fix" such an optimization bug, because the code behind it is valid for ASTs that may have been put into that form from conforming code generated by macros and branches that wouldn't be called. There's nothing to fix.

You're telling it to do something impossible, and it's assuming it can't happen.

Re:C for All by msla on Friday March 23, 2018 @05:17PM (#16662708)
There's also behavior which is undefined for hardware reasons.

An example is what the C standard calls "trap representations": Bit patterns which fit into the space occupied by a specific type, but which will cause a hardware trap (exception, interrupt, what have you) if you actually store them in a variable of that type. The only type which cannot have trap representations is unsigned char. Basically, what it amounts to is this: C compilers don't compile to a runtime, they compile to raw machine code with, perhaps, a standard library. If you do something the hardware doesn't like when your program runs, well, the C compiler is long gone by that point and the C standard makes no guarantees.

More prosaically, storing to a location beyond the end of an array might not cause a segfault. It might corrupt some other array, it might cause a hardware crash, it might even corrupt the program's machine code. Because C is explicitly a language for embedded hardware, with no MMUs, no W^X protection, and no OSes, the C standard can say very little about such things.

Re:C for All by tlb on Friday March 23, 2018 @03:45PM (#16661948)
You're mixing up "undefined behavior" and "implementation-defined" behavior. Implementation-defined behavior is fine. "Undefined behavior", as the term is used in the C spec, means that literally anything can happen. The compiler is allowed to assume that UB never happens, so if it does the program can produce random results.

See http://en.cppreference.com/w/cpp/language/ub

Re:C for All by geezerjay on Friday March 23, 2018 @04:02PM (#16662131)
> You're mixing up "undefined behavior" and "implementation-defined" behavior. Implementation-defined behavior is fine. "Undefined behavior", as the term is used in the C spec, means that literally anything can happen.

Actually I didn't. My point was rather obvious: the whole point of the standards specifying UB is precisely to let implementations define the behavior themselves.

Re:C for All by loup-vaillant on Friday March 23, 2018 @08:10PM (#16663784)
> the whole point of the standards specifying UB is precisely to let implementations define the behavior themselves.

This used to be the case. Signed integer overflow for instance is undefined because some CPUs go bananas when you try that. Other platform performed 2's complement just fine, and we used to be able to rely on this.

No longer.

See, the standard doesn't say "implementation defined". It doesn't say "undefined on platforms that go bananas, implementation defined otherwise". It says "undefined" period.

Signed integer overflow is undefined on all platforms, even your modern x86-64 CPU. Compiler writers interpreted it as a licence to assume it never happens, to help optimisations. For instance:

  int x = whatever;
  x += small_increment;
  if (x < 0) { // check for overflow
      abort(); // security shut down
  }
  proceed(x);  // overflow didn't happen, we're safe!
Here's what the compiler thinks:

  int x = whatever;
  x += small_increment;
  if (x < 0) { // only true if signed overflow -> false
      abort(); // dead code
  }
  proceed(x);
Then the compiler simply deletes your security check:

  int x = whatever;
  x += small_increment;
  proceed(x);
Don't listen to Chandler Carruth, nasal demons are real. Some undefined behaviours can encrypt your whole hard drive, assuming they're exploitable by malicious inputs.
Re:C for All by userbinator on Friday March 23, 2018 @09:45PM (#16664263)
This is the sort of emotionally-charged fearmongering around UB that really makes any discussion pointless. That example is wrong. Integers can be signed. If a compiler cannot prove x >= 0, then it simply cannot remove that code.

Now, if you used

    unsigned int x = whatever;
    ...
    if(x < 0)
There would be an obvious case for removing that if.
Re:C for All by teraflop on Friday March 23, 2018 @11:01PM (#16664546)
A very simple test case demonstrates that GCC can remove tests in the presence of signed overflow, even in ways that change a program's behavior.

    $ cat undefined.c
    #include <limits.h>
    #include <stdio.h>
    #include <stdlib.h>

    int main() {
        int x = INT_MAX;
        if (x+1 > x) {
            printf("%d > %d\n", x+1, x);
        } else {
            printf("overflow!\n");
        }
    }
    
    $ gcc --version
    gcc (Ubuntu 7.2.0-8ubuntu3.2) 7.2.0
    Copyright (C) 2017 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
    
    $ gcc undefined.c && ./a.out
    overflow!
    
    $ gcc -O3 undefined.c && ./a.out
    -2147483648 > 2147483647
Re:C for All by userbinator on Friday March 23, 2018 @11:19PM (#16664630)
Yes, that example is well-known but different; here, the compiler is assuming that x + 1 will always be greater than x, which is entirely something else than the parent's assertion of assuming that x + small_increment will always be positive.
Re:C for All by loup-vaillant on Saturday March 24, 2018 @07:13AM (#16666023)
The difference doesn't matter, you would know better if you weren't clinging so hard to your beliefs. Here's the "difference":

  $ cat undefined.c
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>

  int main() {
      int x = INT_MAX;
      if (x+1 < 0) {
          printf("%d < 0\n", x+1);
      } else {
          printf("overflow!\n");
      }
  }

  $ gcc --version
  gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
  Copyright (C) 2015 Free Software Foundation, Inc.
  This is free software; see the source for copying conditions.  There is NO
  warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

  $ gcc undefined.c && ./a.out
  -2147483648 < 0

  $ gcc -O3 undefined.c && ./a.out
  overflow!
The security check is gone all the same.
Re:C for All by yorwba on Friday March 23, 2018 @11:17PM (#16664616)
Your "small_increment" needs to be at least large enough to turn the smallest negative int into a non-negative number, and that makes it unrepresentable as a signed int itself.

There are cases where the compiler removes misguided overflow checks, since "perform UB, then check whether it happened" doesn't actually work, but your example is not such a case.

Re:C for All by loup-vaillant on Saturday March 24, 2018 @10:12AM (#16666664)
> "perform UB, then check whether it happened" doesn't actually work

This raises an interesting point, because in many cases, assuming 2's complement and wrapping around, checking for overflow after the fact, as opposed to preventing it from happening in the first place, is actually easier. (And actually works if you use the `-fwrap` flag.)

The right thing should be easier to do than the wrong thing. It's a shame this is not the case here.

Re:C for All by loup-vaillant on Saturday March 24, 2018 @07:15AM (#16666034)
I was of course assuming that `whatever` was positive, and the compiler knew it. See this sibling thread: https://news.ycombinator.com/item?id=16664546
Re:C for All by aw1621107 on Friday March 23, 2018 @04:13PM (#16662247)
> let implementations define the behavior themselves

That's literally the definition of implementation-defined behavior.

Undefined behavior really means undefined; in terms of the C language, there are no constraints on behavior. Sure, you might get a result one way on one implementation, but if you rely on that you're technically writing a dialect of C, and need to let the compiler know using flags.

Re:C for All by UncleMeat on Friday March 23, 2018 @04:47PM (#16662501)
Implementation defined behavior does the same thing for a given implementation. The compiler can produce totally different code every time you compile a program with UB. It can even affect code that is totally unrelated to the UB.
Re:C for All by mattkrause on Friday March 23, 2018 @11:59PM (#16664779)
I'm not sure I buy that second part.

Legalistically, I guess it could (the behavior isn't defined, after all), but typically the optimizer makes some valid-only-if-the-code-is deductions and things snowball from there....

Re:C for All by hermitdev on Monday April 02, 2018 @09:18AM (#16734199)
That's not the point, though. In the case of undefined behavior, a compiler doesn't have to define the behavior or even act consistently (most evident by the behavior of the compiler's optimizer).

This is entirely different than implementation defined in that a conforming compiler has to document the behavior they implement and do it consistently.

Re:C for All by kbenson on Friday March 23, 2018 @10:57PM (#16664534)
I'll just refer you to a comment I wrote last week, which I covers a specific case where undefined behavior and how a compiler chose to handle it caused a security problem in Cap'N'Proto.[1] I think it's about as condensed as I've seen it explained, while linking to further information.

1: https://news.ycombinator.com/item?id=16596409

Re:C for All by userbinator on Friday March 23, 2018 @10:11PM (#16664367)
I've noticed the same thing. A strongly adversarial relationship between compiler writers and users, justified by religious adherence to The Holy Standard and a complete lack of understanding of how people actually want and expect the language to behave in practice.

Undefined behavior just refers to something that is not covered by the international standard, and therefore doesn't exist nor should be used, but an implementation may offer implementation-specific behavior

Indeed. Even the standard itself, to quote its definition of undefined behaviour (emphasis mine):

"behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements NOTE Possible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message)."

The fact that the standard "imposes no requirements" should not be taken as carte blanche to completely ignore the intent of the programmers and do entirely unreasonable things "just because you can", yet unfortunately quite a few in the compiler/programming language community think that way.

It's why I'm very encouraging of more programmers writing their own compilers and exploring new techniques, to move away from that suffocating and divisive culture. Compilers should be helping users, not acting aggressively against them just because it's allowed by one standards committee.

Re:C for All by pjmlp on Saturday March 24, 2018 @03:51AM (#16665422)
The thing is that you only see this compiler writer culture among C and C++ devs.

Other language communities that put correctness before performance at any cost, don't share this mentality, including the compiler writers.

Re:C for All by sedachv on Friday March 23, 2018 @06:06PM (#16662997)
> I don't get this "undefined behavior" cliche.

The problem is that you can do everything in portable C that you can do with undefined behavior in C, and often in a more straightforward fashion.[1] The compiler won't tell you that you are doing it wrong; there are many examples, tutorials, and books that encourage you to do the wrong thing. The undefined behavior will work until you switch to a different system. Why allow the wrong thing to continue to happen?

[1] A very good example of this is endianness:

https://news.ycombinator.com/item?id=16189110

https://commandcenter.blogspot.se/2012/04/byte-order-fallacy...

Re:C for All by fjsolwmv on Saturday March 24, 2018 @01:26AM (#16665033)
You are conflating undefined with implementation defined. If you use undefined behavior the compiler might simply delete your code since it's not defined.
Re:C for All by sedachv on Saturday March 24, 2018 @02:22AM (#16665168)
You and a lot of people in this discussion seem confused the other way. The amount of things in C that are implementation-defined is relatively small and most has to do with character encodings and byte size (storage unit). The kind of behavior people are pointing to is definitely labeled as undefined, and there is a lot of it. Take a look at Harbison and Steele for example.
Re:C for All by tytytytytytytyt on Friday March 23, 2018 @05:37PM (#16662822)
Or it's something undesirable and you parroting back the definition of UB doesn't add anything.
Re:C for All by blackguardx on Friday March 23, 2018 @03:39PM (#16661891)
I think many people overlook that "undefined behavior" mainly limits portability. Invoking undefined behavior won't cause your program to do random things, it just might not do the same thing when you switch to a different compiler. This is still a big problem, but not as big as people make it out to be, in my opinion. I think undefined behavior problems in the standard could be fixed, but compilers would need special flags for backwards compatibility with programs that rely on it.
Re:C for All by UncleMeat on Friday March 23, 2018 @04:48PM (#16662518)
> Invoking undefined behavior won't cause your program to do random things

According to the spec, it literally can. The compiler is free to replace your entire program with unrelated functionality. It can ever do a different thing each time it compiles the program.

There are implementation specific behaviors (# of bits in a char), which are different.

Re:C for All by khedoros1 on Friday March 23, 2018 @06:59PM (#16663402)
Undefined behavior is undefined even using the same version of the same compiler. If it does produce the same repeatable behavior, consider it luck.

Implementation-defined behavior limits portability in the way you describe, but UB's not the same thing. IB should behave the same way in the same implementation. UB doesn't have to.

Re:C for All by aw1621107 on Friday March 23, 2018 @04:17PM (#16662284)
> "undefined behavior" mainly limits portability

I had the impression that some things are UB because they can't specify a single behavior that would be efficient across all platforms C targets.

Re:C for All by fjsolwmv on Saturday March 24, 2018 @01:31AM (#16665041)
No that's implementation-defined. Undefined means code that is logically incorrect, but too expensive for the compiler/runtime to check and handle (reject) it. A simple example is array out of bounds access. It's too expensive to demand the compiler prevent it or guarantee to trigger some kind of signal at runtime. But if you are willing to pay the cost , you can use a language or compiler that is safer and promises to throw an exception or provides safe statically sizd arrays