Undefined behavior is one of the tricks that enabled C compilers to catch up with what everyone else was doing in big iron outside AT&T walls.
Fran Allen the Turing awardee on high-performance computing and compiler research, stated this on C:
"Oh, it was quite a while ago. I kind of stopped when C came out. That was a big blow. We were making so much good progress on optimizations and transformations. We were getting rid of just one nice problem after another. When C came out, at one of the SIGPLAN compiler conferences, there was a debate between Steve Johnson from Bell Labs, who was supporting C, and one of our people, Bill Harrison, who was working on a project that I had at that time supporting automatic optimization...The nubbin of the debate was Steve's defense of not having to build optimizers anymore because the programmer would take care of it. That it was really a programmer's issue....
Seibel: Do you think C is a reasonable language if they had restricted its use to operating-system kernels?
Allen: Oh, yeah. That would have been fine. And, in fact, you need to have something like that, something where experts can really fine-tune without big bottlenecks because those are key problems to solve. By 1960, we had a long list of amazing languages: Lisp, APL, Fortran, COBOL, Algol 60. These are higher-level than C. We have seriously regressed, since C developed. C has destroyed our ability to advance the state of the art in automatic optimization, automatic parallelization, automatic mapping of a high-level language to the machine. This is one of the reasons compilers are ... basically not taught much anymore in the colleges and universities."
-- Fran Allen interview, Excerpted from: Peter Seibel. Coders at Work: Reflections on the Craft of Programming
I find that very patronizing.
Yes, we mere compiler users complain about specific examples; that’s because we keep finding specific examples that break in weird ways!
But at the same time, we realize that it’s a general approach that causes trouble. As I understand it: assuming that undefined behavior cannot happen and optimizing on that basis.
I’m not arguing for specific small tweaks to fix a few known weirdnesses, or even a more general adjustment to the notions of poison and undef (as your mention of “massive email threads” suggests).
I’m arguing, first, that the entire approach seems to be misguided and not working well; as evidence, many real-world examples of code doing extremely unexpected things, and the position of the Linux kernel maintainers (surely an important group of users by any standard).
Second, that the problem seems to be largely ignored by compiler writers; as evidence, no changes or improvements, despite several proposals from outsiders. You can point to massive internal debate on email threads but it amounts to nothing if nothing comes out of it.
If you're not a compiler writer, you probably have a mental model of undefined behavior that works like this:
Sure, you probably don't think it's so brazenly stated, but the general sentiment from compiler users tends to be that all you have to do is delete that if statement and everyone's happy.But that's not how compilers work, not in the slightest. Instead, the undefined behavior effects tend to work like a chain: you dereferenced a pointer, therefore it's now safe to assume on any subsequent path that it will always be safe to dereference that pointer, therefore we can move that later load out of that loop even though we don't know if the loop will be executed in the first place. And the reason that logic works is because we're allowed to optimize under the assumption of undefined behavior in one of those steps.
Now in some cases--signed integer overflow and strict aliasing come most readily to mind--there is a single knob you can twist to disable the undefined behavior (and most compilers let you twist that knob with command-line flags). Arguing whether or not the standard itself should dictate a different position on that knob is one thing, and it's by no means a bad discussion to have. But instead arguing "let's let you keep undefined behavior, but let's try to specify what you can do with undefined behavior" runs into the problem that it's very difficult to actually specify what somewhat-constrained undefined behavior really means. Look up the massive email threads on LLVM that try to tease what poison and undef actually mean.