News for hackers. Stuff that matters.
Re:Undefined behavior in C is a reading error by vyodaiken on Thursday May 20, 2021 @03:18PM (#27225762)
The current situation is not good for compiler writers either. But nobody has ever shown that either C programmers want to sacrifice safety for "optimizations", or that these UB optimizations actually improve performance of anything.
Re:Undefined behavior in C is a reading error by ynik on Thursday May 20, 2021 @03:47PM (#27226174)
What do you mean by "these UB optimizations"? C is a low-level language; it's basically impossible for a compiler to reason about the code unless it makes certain assumptions. It needs to assume the code is not self-modifying to do pretty much any code-generation more intelligent than a macro assembler. It needs to assume the code isn't messing with the stack frames/return addresses in order to inline functions. It needs to assume the code isn't using an out-of-bounds pointer to access a neighboring local variable, so that it can move local variables into registers. "gcc -O0" is a good approximation for the performance you get if the compiler isn't allowed to optimize based on UB.

Yes, that means C without optimizing without UB is slower than Java. Optimizations need some form of reasoning about what's happening. For Java it's optimizing based on guarantees provided by the language (there's no raw pointers that could mess with the things listed above). But C doesn't provide any hard guarantees, so instead it needs to blindly assume that the code will behave sanely.

Also note that for many of the more manageable sources of UB, most compilers provide a choice (-fwrapv, -fno-strict-aliasing, ...). Yet few projects use these options, even when they use other gcc/clang-specific features. Doesn't that indicate that C programmers indeed want to sacrifice safety for optimizations?

Re:Undefined behavior in C is a reading error by gpderetta on Thursday May 20, 2021 @04:40PM (#27226917)
Exactly.

For example there were programs 30-40 years ago that relied on exact stack layouts. These days everybody would agree they are completely broken.

The issue of course is that it is extremely hard to write programs that have no UB. It would be nice for compilers to have an option to automatically introduce assetions whenever they rely on some UB-derived axiom, basically as a sort of lightweight sanitizer.

In fact if we had sanitizers 30-40 years ago probably things would be better today.

Re:Undefined behavior in C is a reading error by MauranKilom on Thursday May 20, 2021 @05:49PM (#27227816)
> It would be nice for compilers to have an option to automatically introduce assetions whenever they rely on some UB-derived axiom

Modifying a value from a different thread without synchronization is UB. The compiler assumes this does not happen in order to e.g. move things into registers. Could you elaborate how (and how often) you would like to have this kind of UB-derived axiom ("this value remains the same from here to there") checked with assertions?

Re:Undefined behavior in C is a reading error by gpderetta on Thursday May 20, 2021 @06:15PM (#27228099)
Obviously you wouldn't be able to catch many, or even most cases. Use-after-free is another case that would be very expensive to detect.
Re:Undefined behavior in C is a reading error by pjmlp on Friday May 21, 2021 @04:17AM (#27231638)
I was using it in 2000,

https://www.parasoft.com/products/parasoft-insure/

21 years later it is still an uphill battle to adopt such technology in C and C++ projects.

Re:Undefined behavior in C is a reading error by pjmlp on Friday May 21, 2021 @04:14AM (#27231626)
We had sanitizers since C exists, 1979 to be more exact.

"Although the first edition of K&R described most of the rules that brought C's type structure to its present form, many programs written in the older, more relaxed style persisted, and so did compilers that tolerated it. To encourage people to pay more attention to the official language rules, to detect legal but suspicious constructions, and to help find interface mismatches undetectable with simple mechanisms for separate compilation, Steve Johnson adapted his pcc compiler to produce lint [Johnson 79b], which scanned a set of files and remarked on dubious constructions. "

-- https://www.bell-labs.com/usr/dmr/www/chist.html

Re:Undefined behavior in C is a reading error by gpderetta on Friday May 21, 2021 @04:49AM (#27231818)
I was going to add "in open source compilers" to hedge my statement :).

That seems to be a static analysis tool though (which generally have not been great). Did it also inject runtime checks?

Re:Undefined behavior in C is a reading error by pjmlp on Friday May 21, 2021 @06:17AM (#27232251)
No, but still not even that gets the love it deserve.

And a famous commercial variant of it has been PC-lint from https://www.gimpel.com/.

Being available on open source compilers does little to change the culture, as per latest surveys only 11% of developers care to use any kind of tooling for improving their code quality in C and C++.

At CppCon a couple of years ago, only about 1% of the audience answered positively to Herb Sutters' question.

Re:Undefined behavior in C is a reading error by gpderetta on Friday May 21, 2021 @07:21AM (#27232663)
Those numbers are indeed a bit depressing.
Re:Undefined behavior in C is a reading error by pjmlp on Friday May 21, 2021 @09:02AM (#27233481)
Here is a recent survey, with a more positive number of about 37%, see question 10.

https://isocpp.org/files/papers/CppDevSurvey-2021-04-summary...

Re:Undefined behavior in C is a reading error by vyodaiken on Thursday May 20, 2021 @05:51PM (#27227842)
That's good example, because nobody would complain if stack layouts changed and those programs failed. But if the compiler chooses to "optimize away" checks on stack layout, that's a different thing altogether. Also note that if you use pthreads or Linux clone or you are writing an operating system you can need to rely on exact stack layouts even today.
Re:Undefined behavior in C is a reading error by saagarjha on Thursday May 20, 2021 @09:47PM (#27229644)
Stack layouts are only really relevant at ABI boundaries. In these cases the layout is usually specified in extensions to C or in other ways, such as handwritten assembly.
Re:Undefined behavior in C is a reading error by vyodaiken on Friday May 21, 2021 @06:35AM (#27232364)
Linux clone, pthreads, and os code commonly look at stack boundaries
Re:Undefined behavior in C is a reading error by gpderetta on Friday May 21, 2021 @08:53AM (#27233390)
Not sure what you are referring to with stack boundaries. Of course the ABI imposes some minimal requirements at ABI visible points, but these days you can't even rely on the existence of frame pointers to traverse the stack and you have to use the DWARF unwind machinery. And the content of the stack frame itself is completely unspecified of course.
Re:Undefined behavior in C is a reading error by vyodaiken on Friday May 21, 2021 @02:54PM (#27238325)
So I create a thread with a custom stack which is an allocated buffer. At the top, I write a sequence of bytes in some order. Then I periodically read the top of the stack to see if the stack is getting close to overflow. Meanwhile, the thread code is also addressing the same store.
Re:Undefined behavior in C is a reading error by vyodaiken on Thursday May 20, 2021 @05:33PM (#27227593)
For your last point, the extent of UB driven changes to semantics is still not widely known in the programmer community. Programmers don't read the standard - they read K&R, and K&R is right now describing a different language. We've had 15 years of programmers repeatedly filing bug reports to be told that the expected, tested, relied on, behavior was ephemeral. Only very sophisticated projects figure out about UB.

Of course compilers have to make assumptions. The debate is (a) over what assumptions it is proper to make and (b) what are the permissible behaviors. The false dichotomy: either do without any optimizations at all or accept whatever UB gives you, is not a useful approach.

Re:Undefined behavior in C is a reading error by MauranKilom on Thursday May 20, 2021 @05:46PM (#27227785)
So what optimizations do you mean with "these UB optimizations" then? And would it change your mind to see a benchmark proving the usefulness of that particular UB optimization?
Re:Undefined behavior in C is a reading error by a1369209993 on Thursday May 20, 2021 @10:14PM (#27229796)
> So what optimizations do you mean with "these UB optimizations" then?

Inferring any propositional statement about the program (eg "this pointer is not null") from the fact that its negation would imply undefined behaviour.

Re:Undefined behavior in C is a reading error by vyodaiken on Thursday May 20, 2021 @06:31PM (#27228259)
e.g. assuming UB can't happen: deleting overflow or null pointer checks, deleting comparisons between pointers that are assumed to point at different objects, ...
Re:Undefined behavior in C is a reading error by Dylan16807 on Thursday May 20, 2021 @11:58PM (#27230350)
I'm surprised you don't think deleting a bunch of checks will improve performance.
Re:Undefined behavior in C is a reading error by rectang on Thursday May 20, 2021 @10:06PM (#27229745)
> most compilers provide a choice (-fwrapv, -fno-strict-aliasing, ...). Yet few projects use these options,

Even if you opt into those (and the projects I've been involved with have), the precedent is established: when new compiler optimizations are introduced with disruptive semantics, they are opt-out, not opt-in — a fail-dangerous failure mode.

> Doesn't that indicate that C programmers indeed want to sacrifice safety for optimizations?

Maybe so. Which is one reason I wouldn't call myself a "C programmer" any more. The demands to program responsibly in C are absurdly high.

Re:Undefined behavior in C is a reading error by account42 on Friday May 21, 2021 @08:05AM (#27232974)
> Even if you opt into those (and the projects I've been involved with have), the precedent is established: when new compiler optimizations are introduced with disruptive semantics, they are opt-out, not opt-in — a fail-dangerous failure mode.

Actually, the default for gcc is -O0. You are opting in with -O2 etc.

Re:Undefined behavior in C is a reading error by pjmlp on Friday May 21, 2021 @04:12AM (#27231616)
That is exactly how C should have kept being used, a portable macro assembler, while leaving everything else on the IT stack for more saner languages.

BCPL was anyway designed to Bootstrap CPL, nothing else.

Re:Undefined behavior in C is a reading error by minitech on Thursday May 20, 2021 @07:58PM (#27228965)
> But nobody has ever shown … that these UB optimizations actually improve performance of anything.

That’s just not true. Already examples in this thread: https://news.ycombinator.com/item?id=27223870

Re:Undefined behavior in C is a reading error by rndgermandude on Thursday May 20, 2021 @04:06PM (#27226446)
To do UB "optimizations", the compiler first needs to figure out that there is an UB it can "optimize" anyway. At this point instead of "optimizing" it could, and in my humble opinion absolutely should, blow up the compilation by generating an UB error, so people can fix their stuff.

What about backwards compatibility in regards to a new compiler version deciding to issue errors on UB now? You don't have any guarantees about what happens with UB right now, so if you upgrade to a new version compiler that generates errors instead of "optimizations" everything would be still as before: no guarantees. And it's frankly a lot better to blow up the compilation with errors than to have the compiler accept the UB code and roll a dice on how the final binary will behave later. You can either fix the code to make it compile again, or use an older "known good" version of the compiler that you previously used as a stopgap measure.

I fail to see any reason whatsoever why compilers are still doing all kinds of stupid stuff with UB instead of doing the right thing and issuing errors when they encounter UB.

I also fail to see why the C language designers still insist on keeping so much of the legacy shit around.

Re:Undefined behavior in C is a reading error by nullc on Thursday May 20, 2021 @04:32PM (#27226813)
> To do UB "optimizations", the compiler first needs to figure out that there is an UB it can "optimize" anyway.

The compiler assumes UB will never happen and it makes transformations that will be valid if there happens to be no UB. This doesn't require any explicit detection of UB, and in some cases UB or not is simply undecidable at compile time (as in no compiler could detect it without incorrect results).

Without these assumptions the resulting compiled code would be much slower, though some optimizations have different danger vs speed impact and there certainly can be a case that there are some optimizations that should be eschewed because they're a poor trade-off.

There are many cases where current compilers will warn you when you've done something that is UB. It's probably not the case that they warn for every such detectable case and if so it would be reasonable to ask them to warn about more of them.

I think your irritation is just based on a misunderstanding of the situation.

Compiler authors are C(++) programmers too, they also don't like footguns. They're not trying to screw anyone over. They don't waste their time adding optimizations that don't make real performance improvements just to trip up invalid code.

Re:Undefined behavior in C is a reading error by rndgermandude on Thursday May 20, 2021 @08:37PM (#27229217)
Yes, some UB are not decidable at compile time, but a lot could be easily speced to have a defined behavior at runtime, such as overflows.

The main reason to not spec these things is because people would be arguing "this makes compiled code on my esoteric 9-bit 1-complement chip slower" or "there was this chip in the 70s that did things differently" or "but a short int on Cray was 64-bit". Great, so now the spec has avoidable unnecessary undefined behavior all over the place, and the code other people wrote still does not run correctly on your 9-bit chip. Brought to you by the same people who decided "NULL is not necessarily (void*)0", and who define those integer types everybody uses (instead of stdint) with an "at least this big".

Yes, a lot of that is legacy stuff and was added to accommodate and model things that already existed (the wrong way to go about it, IMO, but hindsight is 20/20), but that's my argument: fix this stuff once and for all and for good in an upcoming spec iteration.

>Without these assumptions the resulting compiled code would be much slower

In some cases, this is true (for different levels of "much slower"), but the trade off here is still "running code that works, but a little slower" vs "running code that does not work and will launch a nuclear strike at Switzerland by accident, but really fast".

In a lot of cases, it will not be slower, or at least not much slower.

>I think your irritation is just based on a misunderstanding of the situation.

Frankly, not really. I started writing my first C (and C++) in the early 90s, and I think I do understand the situation pretty well by now. But I should have been more precise in my initial ranting comment, I give you that.

>They're not trying to screw anyone over.

I didn't say that they are.

Re:Undefined behavior in C is a reading error by saagarjha on Thursday May 20, 2021 @09:52PM (#27229673)
Note that (void *)0 is always NULL, as mandated by the standard.

But, to address the content of your comment: defined behavior at runtime is not necessarily good behavior at runtime. Defining signed integer overflow to wrap, for example, is probably a bad idea, because this is rarely the intent of the code. Having all such operations trap might be a good idea, but now you're going to get the same "stop breaking my working programs" people angry at you.

Re:Undefined behavior in C is a reading error by rndgermandude on Friday May 21, 2021 @02:54AM (#27231225)
Yes, thankfully at least with NULL they didn't fall into the legacy trap and messed up the standard with non-zero NULL that some machines before have been kind of using.

>Defining signed integer overflow to wrap, for example, is probably a bad idea

I wouldn't call it great behavior, but it's at least what most people expect will happen, and most people will be able to understand what's going on, and it's fast on most systems that matter. However, it's still undefined behavior. Just codifying overflows to be wrapping, would therefore be an improvement in my opinion, at least over what we have today.

Re:Undefined behavior in C is a reading error by gpderetta on Friday May 21, 2021 @04:55AM (#27231838)
I would say that most people expect it not to happen. If they really had to mandate a behavior, it should be to trap.
Re:Undefined behavior in C is a reading error by account42 on Friday May 21, 2021 @08:11AM (#27233025)
> Note that (void *)0 is always NULL, as mandated by the standard.

Also note that this is distinct from the memory representation of (void *)0 being all 0 bits, which is explicitly not mandated.

Re:Undefined behavior in C is a reading error by gpderetta on Thursday May 20, 2021 @04:32PM (#27226807)
> To do UB "optimizations", the compiler first needs to figure out that there is an UB it can "optimize" anyway.

That's not how compwillrs work. In fact in the general case it is impossible to figure out at compile time that "there is an UB".

The compiler instead assumes as an axiom that no UB can ever happen and uses the axioms to prove properties of the code.

These days if you want to catch UB, compile with -fsanitize=undefined-behaviour. The program wll then trap if UB is actually detected at runtime.

Re:Undefined behavior in C is a reading error by IcePic on Friday May 21, 2021 @02:31AM (#27231116)
> These days if you want to catch UB, compile with -fsanitize=undefined-behaviour. The program wll then trap if UB is actually detected at runtime.

So, let me get this straight, someone wants to make sure pointer p is not null (in the wrong way), and codes something like the examples in posts above like if (!p) ... and if that doesn't trigger calls use(*p), but compiler decides p can never be null because that would be UB and hence removes the check.

The C coder dumps the code and gets upset because the check is removed and gets the hint to catch UB by adding -fsanitize .. that "catches UB" in the above scenario so that the program will "trap if UB is detected".

I think we just came full circle there.

Sure, the -f will catch ALL detected bugs and so on, but I still found it a bit funny.

Re:Undefined behavior in C is a reading error by gpderetta on Friday May 21, 2021 @04:53AM (#27231830)
It is a bit different.

Ubsan will abort an invalid program if it detect ub. It doesn't let you handle it. So you shouldn't remove the erroneous check, but fix it so it is no longer erroneous, and ubsan will help you identify these errors.

Also ubsan adds significant overhead so it is not really appropriate for production builds unfortunately (hence my wish for a less powerful ubsan-lite but with lower overhead).

Re:Undefined behavior in C is a reading error by tom_mellior on Friday May 21, 2021 @04:00PM (#27239178)
I think you are misunderstanding the situation. Given code like:

    if (!p) {
        use(*p);
    }
(given no previous knowledge about p) no compiler will remove the "if (!p)" part.

What people are complaining about is the opposite case:

    use(*p);

    /* The compiler reasons that if p == NULL, the program would have crashed by now,
       so if we got here, p != NULL must hold. */

    if (!p) {  // the compiler can remove this branch
        report_error();
    }