Now, to see why this might actually produce a bug in working code--say some other thread has access to the not-null value and sets it racily (non-atomically) to null. Or (since most compilers are super conservative about checks of values that escape a function because they can't do proper alias analysis), some code accidentally buffer overflows and updates the pointer to null while intending to do something else. Suddenly, this obvious optimization becomes invalid!
Arguments to the effect of "the compiler shouldn't optimize out that loop due to assuming absence of undefined behavior" are basically arguments for compilers to leave tons of performance on the table, due to the fact that sometimes C programs don't follow the standard (e.g. forgetting to use atomics, or indexing out of bounds). While it's a legitimate argument, I don't think people would be too happy to find their C programs losing to Java in benchmarks on -O3, either.
I seriously would accept everything running at half speed for the certainty of not being subject to the problems of C level bugs. But as Rust grows in popularity, it looks like I won't need to worry about that.
I think most people would. But the described code is still buggy even when it's not optimized.
Anyway this is all fine, but as you can imagine you lose a lot of optimizations that are facilitated by all that UB, so the compiler authors should then counter with some way to signal that you want the original undefined semantics (for instance, references in C++ and restrict pointers in C), or provide compile-time checking to prevent misuse that messes up optimizations (e.g. Rust's Send+Sync for avoiding data races, or UnsafeCell for signaling lack of restrict semantics / raw pointers for lack of non-nullability).
The last sentence is not true. If there is UB inside the if, the compiler may assume that the if condition never evaluates to true (and hence delete that branch of the if), but it may certainly not remove the surrounding loop (unless it can also prove that the condition must be true).
This is…complicated. Let's say you have an array of ten numbers, and then you take user input and use that to index into the array. This program is well-formed…as long as the user never inputs a number beyond ten. If they do, then the program is invalid. In general, the presence of undefined behavior is an attribute of the running program, not the source code itself. If there exists any execution where only defined behavior, the compiler may not deviate from the standard. However, what you probably meant is behavior in the face of the existence of runtime undefined behavior, in which case you are correct that a compiler could write clairvoyant code that refuses to execute the first instruction if it knows that UB will happen at some point in the program.
Is that still the case for current C standards, or did something change in C99/C11?
In C89 there was a list of PERMISSIBLE things that compilers could do upon encountering undefined behavior. In C99 that was changed to a list of POSSIBLE things. And compilers have taken full advantage of that.
As pointed out here and in the HN comments on that article, the phrase "ignoring the situation completely with unpredictable results" is present in both those standards, and is arguably what allows aggressive compiler optimizations to be made, since to a first approximation those optimizations rely on ignoring control flow that encounters UB.
[0]: https://news.quelsolaar.com/2020/03/16/how-one-word-broke-c/
But the first possibility listed is "ignoring the situation completely with unpredictable results". Surely that covers any possible consequences.
Absolutely not. In the C89 standard, undefined behavior becomes undefined *UPON USE OF* the thing that is undefined. In current compilers, the existence of undefined behavior anywhere in your program is an excuse to do anything that the compiler wants to with all of the rest of your program. Even if the undefined behavior is never executed. Even if the undefined behavior happens after the code that you have encountered.
So, for example, undefined behavior that can be encountered within a loop makes it allowable to simply remove the loop. Even if the undefined behavior is inside of an if that does not happen to evaluate to true with your inputs.