News for hackers. Stuff that matters.
Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by naasking on Friday April 21, 2017 @09:58AM (#14165658)
> I am no expert, but what I understood is that the C standard defines some situations (such as an overflow error) which result in "undefined behaviour". In this situation, the compiler is free to do whatever he wants.

Except the proper answer to this scenario is to refuse to compile with an appropriate error, not to eliminate your code.

Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by zAy0LfpBZLC8mAC on Friday April 21, 2017 @01:42PM (#14167618)
You are asking for the impossible. It is not generally possible for the compiler to detect whether undefined behaviour can occur.

Take a function that returns the square of a signed integer: As long as it's called with arguments that are small enough, the behaviour is perfectly defined. But the compiler cannot necessarily know which values might get passed to it at runtime.

Now, it could in principle add runtime checks that do something defined, like abort the program, when the preconditions for defined behaviour are not met--but that often makes the generated code slow.

That is why the standard says that the compiler can do anything it wants in the case of undefined behaviour: It's just a different way of saying that it's the responsibility of the programmer to make sure that the preconditions for defined behaviour are never violated, so the compiler can generate code based on the assumption that that is the case, instead of littering the code with tons of overhead for the case that the programmer did something wrong.

Compilers don't detect undefined behaviour and then use those detected instances of UB in order to mess up your software, they simply operate on the assumption that there is no UB in your code--which means that the code they generate is correct if there is indeed no UB, but there are no guarantees as to what happens if you don't hold up your implicit promise to not invoke UB.

Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by kbenson on Friday April 21, 2017 @02:35PM (#14168093)
Any time a compiler has elided some code it's because it has detected that it can because it's undefined, so in the context of this argument saying that the compiler cannot determine undefined behavior is not useful, as we are only talking about the times it already has.

The obvious interpretation of the request to have the compiler error is to have the compiler error in any case it would elide statements. That may be impractical in many or most cases, but it's a different problem than what you are talking about. Presumably there would be some cases where the compiler could warn or abort (if a flag was used) if branches were elided based on undefined behavior and those same branches also included an abort of some type shortly thereafter. Or maybe some better heuristic that I'm not thinking of.

Hard is not the same as impossible, and imperfect is often better than nothing.

Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by zAy0LfpBZLC8mAC on Friday April 21, 2017 @06:14PM (#14169852)
> Any time a compiler has elided some code it's because it has detected that it can because it's undefined, so in the context of this argument saying that the compiler cannot determine undefined behavior is not useful, as we are only talking about the times it already has.

No, you have it all backwards, and that's not what's happening. The compiler doesn't elide code because it is undefined. The compiler elides code because it would never be executed unless it happens to be called with arguments that would produce undefined behaviour anyway.

Take this, for example:

    void foo(int *x){
      *x=5;
      if(x==NULL)abort();
    }
The operation that is potentially UB here is the *x=5--if x happens to be NULL. But the compiler doesn't remove that assignment, it removes the check following it. And it does so because execution could never reach that check in a defined manner unless it is false: Either x is NULL, then the behaviour of the assignment is undefined (and commonly would cause a segfault, for example), or it is not NULL, then the check is superfluous.

And there is nothing necessarily wrong with that code: It could be (and such code is common) that it is actually never called with a NULL pointer, in which case the behaviour is perfectly defined.

If a compiler actually determines that your code will always exhibit undefined behaviour, then chances are it will indeed warn you, but that just isn't what usually happens, and it's not the cause of such bugs. The compiler doesn't say "this is undefined, therefore, let's screw it up", it's exactly the opposite: It says "if we assume that this code is never called with arguments that would cause undefined behaviour, what is the most efficient machine code that we could map it to?"

> The obvious interpretation of the request to have the compiler error is to have the compiler error in any case it would elide statements.

That would be plain idiotic. It is perfectly normal to have tons of dead code, erroring out in that case would just make it impossible to compile anything.

You have to consider that the compiler isn't interested in the elegant structure of your code, so it mashes it all up to figure out the most efficient machine code for the whole thing. So, you might have code that calls lots of functions on an array, say, where each function does a bounds check. Now, the compiler potentially will inline it all into one big spaghetti function. And then it will figure out that many of the bounds checks are actually redundant (again, under the assumption that the code doesn't invoke undefined behaviour), so it will remove any checks and corresponding error handling paths that it can prove to be implied by preceding checks, or it might try and combine multiple checks into one.

That does not mean that there is anything wrong with your code, or that you should manually remove all the redundant checks (which might not be redundant for other call sites, after all). The code is perfectly fine, and the compiler tries its best to remove anything that you don't actually need.

> Hard is not the same as impossible, and imperfect is often better than nothing.

True, but also besides the point. Compilers do issue warnings for lots of stuff, and more are added regularly, but much of what you are suggesting is actually not in any way even a coherent idea, and as such, is indeed not just hard, but impossible.

Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by kbenson on Friday April 21, 2017 @08:37PM (#14170508)
> No, you have it all backwards, and that's not what's happening. The compiler doesn't elide code because it is undefined. The compiler elides code because it would never be executed unless it happens to be called with arguments that would produce undefined behaviour anyway.

You're right, it's the interaction between the undefined behavior and the ability to reason about what's possible that's the problem. It's also not entirely pertinent to the overall point, which is that the compiler does know when it elides code, and optimizing out instructions is specifically what this particular issue is about.

> That would be plain idiotic. It is perfectly normal to have tons of dead code, erroring out in that case would just make it impossible to compile anything.

The compiler doesn't need to error on all instances of this, but a flag to have the compiler warn or error if it would remove a unique branch and that branch may exit prior to returning control, that could trigger the desired behavior. If the compiler has enough knowledge to determine redundant code, then it has enough knowledge to know whether it is removing code that is not due to duplication.

C compilers have historically chosen to error on the side of speed, not on the side of safety, and we've built ourselves into a corner. Any new language introduced today that said "well, there's some interesting interactions sometimes if you don't pay close attention, and the compiler/VM might remove statements you write because they are testing those same weird interactions[1] as it things they can't happen" would be laughed out of town.

That the optimizations that C compilers do are complex and have many stages of optimization is not a suitable counter for the criticism that those same optimizations sometimes cause non-obvious interactions with safety checks meant to test the same edge cases that those optimizations take advantage of. That's like someone saying for safety reasons you need to see at least 20 feet of road in front of you at all times per 10 MPH on the highway, and people complaining about how that's not feasible because it would force you to slow down 10-20 MPH occasionally as you went around some turns. Yes. Yes it would. Just because you can do an optimization, doesn't mean you should.

> much of what you are suggesting is actually not in any way even a coherent idea

I'm not the original commenter, but if you're referring to my suggestion of at least warning when entire unique branches of the original code are removed, that's entirely possible. That it might require reworking or ever removing portions of the current optimization pipelines, or cause compilation speed to slow considerably is irrelevant to this particular aspect, because right now we aren't having a discussion of whether it's worth it, but whether it's event possible.

1: As happened here in this case.

Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by zAy0LfpBZLC8mAC on Friday April 21, 2017 @10:37PM (#14170854)
> The compiler doesn't need to error on all instances of this, but a flag to have the compiler warn or error if it would remove a unique branch and that branch may exit prior to returning control, that could trigger the desired behavior. If the compiler has enough knowledge to determine redundant code, then it has enough knowledge to know whether it is removing code that is not due to duplication.

I wrote "redundant", not "duplicate", intentionally. A check is redundant if other code implies that it cannot possibly ever end up true. It being a duplicate check is not the only way for that to happen, and it's actually the exception. Also, no, the compiler most likely doesn't have that knowledge. A compiler doesn't work the way you seem to think.

> Any new language introduced today that said "well, there's some interesting interactions sometimes if you don't pay close attention, and the compiler/VM might remove statements you write because they are testing those same weird interactions[1] as it things they can't happen" would be laughed out of town.

Or, more likely, it wouldn't. These are good reasons to not write high-level software in C, but there are also good reasons why people who actually need high speed still do use C. And it's not that we like the fact that writing correct C is hard.

You seem to imply that there is no real reason for C behaving the way it does, and that those rules for undefined behaviour only exist to make the life of programmers miserable. Those things are undefined because making them defined would actually be expensive in terms of performance.

> That the optimizations that C compilers do are complex and have many stages of optimization is not a suitable counter for the criticism that those same optimizations sometimes cause non-obvious interactions with safety checks meant to test the same edge cases that those optimizations take advantage of.

The way you phrase things suggests that you might be confused about how the compiler "reasons". The compiler doesn't read your program, sees what you mean, and then tries to find loopholes in order to misunderstand you. The compiler reads your program, and only understands what your program means according to the formal specification of the language that you claim it is written in. In the example I gave above, you seem to think that there is a call to abort() that the compiler "removes". There isn't. According to the C spec, that call is unreachable, and as such the semantics of that statement is a noop, which is what the compiler will correctly map to machine code, somehow. Explicit dead code removal on some intermediate representation of the AST is just one way that could happen, and it's an implementation detail of the compiler.

> That's like someone saying for safety reasons you need to see at least 20 feet of road in front of you at all times per 10 MPH on the highway, and people complaining about how that's not feasible because it would force you to slow down 10-20 MPH occasionally as you went around some turns. Yes. Yes it would.

Yep, that's a perfect analogy for people complaining that when they talk to a C compiler, they maybe should be writing C, and not a language that they themselves made up, if they expect the compiler to understand them. Even it's sometimes difficult.

> Just because you can do an optimization, doesn't mean you should.

I agree. But for the most part, that's not what's happening. For the most part, optimizations are not intended to break your code, but rather it so happens that, in order to optimize some code, you have to rely on all code having certain correctness properties (that it should have if it is C code, according to the C spec), which then, unfortunately, happens to break some code that doesn't have those properties. Often it's somewhere between infeasible and impossible to distinguish those cases that are correct and can thus be optimized without introducing unwanted behaviour from those that are not correct and thus break as a result.

> I'm not the original commenter, but if you're referring to my suggestion of at least warning when entire unique branches of the original code are removed, that's entirely possible. That it might require reworking or ever removing portions of the current optimization pipelines, or cause compilation speed to slow considerably is irrelevant to this particular aspect, because right now we aren't having a discussion of whether it's worth it, but whether it's event possible.

First, equivalence between pieces of code is undecidable, and second, "unique branches" is not in any way a useful concept anyway.

Sure, you can try to make a compiler detect certain instances of what seems like a safety check that doesn't ever trigger. But that will either be completely ineffective (as it only detects a small minority of cases), or it will produce tons of bogus warnings (because there are tons of cases where it is perfectly sensible to have "unique branches" in your code that are provably never taken, even ones that abort the program, and it is logically impossible to distinguish those from ones that were written with the intent to catch a runtime exception that the programmer expects to actually happen at runtime).

Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by kbenson on Friday April 21, 2017 @11:26PM (#14170966)
You seem to be arguing "It's not possible to catch all cases of this". I'm arguing "It's possible to catch some cases of this, and those should be caught and fixed".

In this specific case, the compiler inferred that since two pointers were added, and pointers that overflow is undefined, that it couldn't possibly be that the programmer specified undefined behavior, so it must be that there was no overflow. Since there was "no overflow", the compiler decided that the conditional statement testing for overflow could never be true, and removed that branch.

In the specific example submitted to HN, the compiler has a few options:

First, it can assume the programmer is infallible and will never make a mistake, and use that to actually change the code as defined for performance reasons.

Second, it can assume the programmer is fallible, and that without additional information, it is unsafe to alter the code based on assumptions of programmer infallibility for safety reasons.

Finally, it can assume the programmer is fallible, but try to keep the information present in some manner (tag as "maybe true") so that it can be reported on later, while not using it for additional optimizations for safety with advantages. It would be trivial to note optimization that it could do, but will not because it cannot reason adequately about one or more required assumptions. It would also be trivial to note occurrences of statements where undefined behavior it knows may be encountered if the programmer is not vigilant about the inputs using that same data (that is, it need not report everything, just what it can definitively find). That is, it's trivial because the hard work of detecting the problem cases is already being done.

Currently many C compilers default to optimizing for performance, the first options above. They could, and many think should at a minimum default to safety instead. We know people aren't infallible, so acting like they are has no basis in reality.

That's not to say every optimization has to be thrown out. There should be a distinction between something that can be assumed because of mathematical properties and constraints the compiler enforces compared to constraints it's assumed the programmer will correctly follow. There are clearly cases where the compiler can optimize based on knowledge it it has. If you cast an unsigned char to an int, and add another unsigned char to it prior to doing any other operation, you can assume there will be no overflow. You can assume the same of short on platforms where int is twice as bit as a short.

The bottom line is that compilers are assuming truthfulness of expressions that aren't necessarily true, and that require extraordinary effort from programmers to make sure they avoid, as evidenced by their continual discovery in enterprise grade libraries and applications.

Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by zAy0LfpBZLC8mAC on Saturday April 22, 2017 @01:43PM (#14173659)
> First, it can assume the programmer is infallible and will never make a mistake, and use that to actually change the code as defined for performance reasons.

You keep saying that it changed the code. It didn't. It compiled exactly what the code said. You dislike C, and that's OK, but that doesn't mean that a C compiler compiling C code into machine language that is semantically equivalent to the C code is somehow changing the program. It's not.

> That is, it's trivial because the hard work of detecting the problem cases is already being done.

No, it's not, you are still committing the same fallacy as before. The compiler doesn't "collect knowledge about undefined behaviour in the program", because that is useless knowledge for the compiler.

The compiler collects knowledge that helps it reason about the program. A big part of that is tracking the range of values variables can take on. That is information that is useful in selecting how to express certain code in machine code. Undefined behaviour plays into this because operations that are defined to have undefined behaviour have no results that need to be considered as possible values of the variable that the result is stored in. So, if the compiler sees an if(x>0&&y>0){ x+=y;x/=2; }, with x and y being ints, it can derive that x will be positive, and therefore, for example, the division can be compiled to a right-shift.

There is no code in the compiler that goes "well, there is this pointer dereference, so let's remove the NULL-check branch". It's rather that the dereference limits the range of possible values of the pointer to non-NULL values, which a later stage then uses to determine that the branch cannot ever be taken, and thus can be eliminated. The same "knowledge" could be inferred from an assignment of a constant, for example, or from a preceding check ... the compiler tracks the value, not whether undefined behaviour could happen.

> Currently many C compilers default to optimizing for performance, the first options above. They could, and many think should at a minimum default to safety instead. We know people aren't infallible, so acting like they are has no basis in reality.

That's completely tautological. Every programming language assumes the programmer to be infallible. Every compiler and interpreter does what the program means according to the language spec, and if the programmer fails to express what they mean in the language, the program will do the wrong thing.

Now, there is an argument to be had over what kind of semantics of a language are easier to reason about than others, and to construct languages that make reasoning about the code as easy as possible, and some of that can be applied to adding additional restraints in a C compiler on top of the language specification, to make the language as understood by the compiler easier to reason about (while still staying within what the C spec defines, so as to stay compatible with existing, correct C code).

But there are two major problems with your reasoning here:

(1) Distuinguishing programming mistakes and legitimately optimizable code is far harder than you think. You are just handwaving through that part, but that's actually the hard part. You will either miss a lot of optimization opportunity, or you will catch close to none of the relevant mistakes. If you think you have the solution to tell those cases apart much better than current compilers do, please write a paper about it, compiler writers certainly will be interested.

(2) Performance is actually kindof central to C. If you don't need performance, you probably should just not be writing C in the first place. And if you actually need performance, just erring on the side of safety isn't necessarily gonna cut it. The question is not whether you could add all of the safety features of, I dunno, python, to C. The question is what you would expect the result to look like? There is a reason why C code tends to be faster than python, and part of that is the lack of safety.

To maybe get an idea of why compilers do assume signed overflow to be undefined behaviour, this article seems to give a good overview: https://kristerw.blogspot.de/2016/02/how-undefined-signed-ov...

Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by kbenson on Saturday April 22, 2017 @02:49PM (#14174048)
> You keep saying that it changed the code. It didn't. It compiled exactly what the code said.

From the submitted article: Thus, the compiler removes this part of the check. It did not compile what the code said, it compiled what it determined it needed to compile. That determination included an assumption of what values a variable could be based on whether they could overflow, which would be undefined. That's the whole point.

The code specfied to test whether "target < segmentStart", and the compiler determined that could never be true and removed that check. We have in this bug report direct evidence that the compiler was too aggressive in its assumptions, as it is indeed possible. It was too aggressive for exactly the reasons I have been going over, which is to say that condition can only never be true as long as the programmer protects the actual values being used from being large enough to cause an overflow in the prior statement.

> It's rather that the dereference limits the range of possible values of the pointer to non-NULL values, which a later stage then uses to determine that the branch cannot ever be taken, and thus can be eliminated.

The check in question (in the compiler in question) specifically uses an assumption that the programmer will prevent an overflow which would be undefined. That is the assumption of infallibility I'm referring to.

> That's completely tautological. Every programming language assumes the programmer to be infallible.

No, they don't. If they did, Java, Rust and just about every dynamic language would never do bounds checks. One of the main reasons for a type system is to force the programmer to follow rules to prevent mistakes.

> Distuinguishing programming mistakes and legitimately optimizable code is far harder than you think.

At this point I'm referring to a specific type of optimization that they are doing that I think rests on shaky presumptions.Removing that optimization is not hard work. It may be hard for the community to stomach, depending on how much performance impact it has.

> Performance is actually kindof central to C. If you don't need performance, you probably should just not be writing C in the first place. And if you actually need performance, just erring on the side of safety isn't necessarily gonna cut it.

I gave a specific example for how to get the same performance if this particular type of optimization was made more conservative. Performance is important, but when it comes to performance or correctness, correctness should win. Full stop.

Very little of what I'm referring to at this point is theoretical. I'm referring to real world situations, mostly the one these comments are in response to. Your comments seem to indicate you think this situation isn't possible. Can you clarify on whether you think the bug report is wrong, or whether I'm incorrect in my assessment of what the bug report is saying, or whether I'm misinterpreting your point? At this point, I'm under the impression that much of what I'm stating is fact, so I'm not sure how to interpret statements such as "It compiled exactly what the code said." as anything but wrong, but that's not getting us anywhere.

Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by zAy0LfpBZLC8mAC on Saturday April 22, 2017 @04:14PM (#14174493)
> It did not compile what the code said,

Yes, it did--that seems to be your fundamental confusion.

What that code means is determined by the specification of the C language, and only the specification of the C language. You constantly keep implying stuff that you think, or hope, or would prefer the code means, but that is completely irrelevant for the question of what the code actually means. Just because some intuitive reading of the characters that make up the code makes you assume that it should means a certain thing, does not make it so.

That code does not mean "check for overflow", no matter how much you wish it did. And because it doesn't mean that, the compiler didn't translate it as that either.

> No, they don't. If they did, Java, Rust and just about every dynamic language would never do bounds checks. One of the main reasons for a type system is to force the programmer to follow rules to prevent mistakes.

You are completely missing the point, essentially due to the same confusion as above. I didn't say that those languages didn't have bounds checks. I said that they assume that the programmer is infallible. Every programming language specifies exactly what each syntactic construct means, and which syntactic constructs don't mean anything, and what the runtime behaviour is, and where it is undefined. That is what makes a programming language a programming language. It is the job of the programmer to translate what they mean into the syntax of the respective programming language. If the programmer makes a mistake in this translation, the programm will be wrong, and it will not do what the programmer meant it to do, no matter which programming language they are using--in that sense, every programming language expects the programmer to be infallible.

The difference between programming languages is not whether they allow you to make mistake (none does or ever will), but how difficult it is (mentally) to avoid making mistakes.

> I gave a specific example for how to get the same performance if this particular type of optimization was made more conservative. Performance is important, but when it comes to performance or correctness, correctness should win. Full stop.

That's completely besides the point. Nobody is saying we should have incorrect code (well, ok, some misguided people probably do, but they aren't really part of this discussion). The question is how we are going to achieve that, and that is ultimately a question of economics: What is the easiest/cheapest way to get the greatest amount of software into a state where its execution matches what the programmer intended? Just claiming that we should throw infinite resources at the problem doesn't actually help the problem disappear.

> Can you clarify on whether you think the bug report is wrong, or whether I'm incorrect in my assessment of what the bug report is saying, or whether I'm misinterpreting your point?

Really none of those, I think. I think the way you think about the problem is just confused, which makes it difficult to nail down why exactly your suggested solutions aren't really solutions.

> I'm under the impression that much of what I'm stating is fact, so I'm not sure how to interpret statements such as "It compiled exactly what the code said." as anything but wrong, but that's not getting us anywhere.

I hope I maybe managed to explain that above? I think that's really at the core of your confusion: You are mixing up what you intuitively think things mean and what things mean according to the appropriate formal definition in the respective context. But code in particular does not mean anything, except for what the formal specification of the respective language defines, and that can deviate arbitrarily far from your intuitive understanding.

It's a bit like false friends in natural languages: Just because you know a word from one language, doesn't mean the same word cannot mean something completely different in another language, and it's just confused to use the vocabulary of one language to determine the meaning of a sentence in a different lanugage.

Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by kbenson on Saturday April 22, 2017 @05:39PM (#14174975)
> Yes, it did--that seems to be your fundamental confusion.

No, it compiled what it determined it had to, based on the C standard. There is a difference. The code, as written, specified a certain set of actions to be taken. The compiler determined some of those directions need not me translated to machine code, and thus did not, but they were specified nonetheless.

To say that the compiler did not remove any code, or directions to be carried out, when translating to machine code, is to subscribe to a torturous and unuseful definition of the terms we have been using.

Of the actions specified by the programmer in the source file, one of which was optimized out in the translation of that source specification to machine code. This change alters the execution path of the program when it is present, and to such a degree that without the optimization the program would halt almost immediately, but with the optimization it allows an out of bounds memory access.

We are not arguing whether the C standard allows this. We are arguing whether the C compilers should do this. There is a distinct difference. Stating that no code was removed has been extremely unhelpful to this conversation, regardless of whether you think it is a technically correct statement. In the generated machine code, a condition of a branch statement does not exist in the version with optimization, but does without it.

The fact that this particular optimization relied on a case where the programmer specified a statement that depending on values not knowable to the compiler at the time of compilation may have resulted in undefined behavior or not makes this a poor optimization to carry out.

> Just because you know a word from one language, doesn't mean the same word cannot mean something completely different in another language, and it's just confused to use the vocabulary of one language to determine the meaning of a sentence in a different lanugage.

Perhaps you could actually address a point I've made instead of arguing over the words used. You are arguing over a technicality of the instead of the topic at hand.

Feel free to reply, I'll read it, but I'm done with this conversation beyond that.

Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by zAy0LfpBZLC8mAC on Saturday April 22, 2017 @09:33PM (#14175925)
> The code, as written, specified a certain set of actions to be taken.

That's a nonsensical statement. "The code, as written" doesn't have any meaning, other than perhaps what you make up in your mind, which is not a useful reference for discussion, unless you also explain what you interpret it to mean.

I understand that maybe you do not actually mean this literally, and that you maybe are just using somewhat imprecise language to get the idea across--the problem is that exactly in the details that you are not spelling out are the problems that this discussion is all about.

> To say that the compiler did not remove any code, or directions to be carried out, when translating to machine code, is to subscribe to a torturous and unuseful definition of the terms we have been using.

No, quite to the contrary. Those definitions might not be useful for day-to-day programming work, but they are exactly the definitions that you need to clearly discuss compiler behaviour, because those are the definitions that the compiler is using, and the compiler is using those definitions because they match the concepts of how you build a compiler.

> Of the actions specified by the programmer in the source file, one of which was optimized out in the translation of that source specification to machine code. This change alters the execution path of the program when it is present

No, there is no "change", that's just confused language. There is a difference between compilation results, but neither of those is in any way the "real" thing, while the other is "changed", they are both equally valid mappings from C to machine code, with one arguably being closer to the intention of the programmer and thus maybe more useful in this specific case.

> We are not arguing whether the C standard allows this. We are arguing whether the C compilers should do this.

The problem is that those are inextricably interlinked, because the compiler must still stay within the bounds of the standard, and still produce code with reasonably good performance.

> Stating that no code was removed has been extremely unhelpful to this conversation, regardless of whether you think it is a technically correct statement.

The point is not that it's a technically correct statement, the point is that that's not necessarily how the compiler "thinks", so it's often unhelpful in discussing compiler behaviour to talk about "removing code".

> In the generated machine code, a condition of a branch statement does not exist in the version with optimization, but does without it.

It just so happens that in this case, the compilation result without optimization was closer to the programmer's intention than with optimization. But the usefulness of this observation is severely limited because in other cases the exact opposite could be true. The programmer wrote something different than what they meant, and the compiler in some situation produced code that still matched the intention of the programmer ...

> The fact that this particular optimization relied on a case where the programmer specified a statement that depending on values not knowable to the compiler at the time of compilation may have resulted in undefined behavior or not makes this a poor optimization to carry out.

Except that if a C compiler avoided all optimizations for which this is true, a lot of code would be a lot slower. You seem to only be seeing some specific cases for which the performance difference is negligible, and the risk of the optimization is obvious to you, and your imprecise use of language doesn't make discussing this any easier. What you don't seem to realize is how much optimization a C compiler does that is perfectly safe that the compiler cannot easily, if at all, distinguish from this arguably dangerous case, which is why the compiler could only choose to either in many cases produce unnecessarily slow code, or use the current strategy and occasionally produce code that does something else than what the programmer had in mind.

> Perhaps you could actually address a point I've made instead of arguing over the words used. You are arguing over a technicality of the instead of the topic at hand.

Your point is incoherent because you are using imprecise language, which makes it difficult to address. That's why I am addressing your imprecise use of language first.

Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by klodolph on Friday April 21, 2017 @01:22PM (#14167445)
I think there's a misunderstanding here—the undefined behavior occurs at runtime, not at compile time, so rejecting it outright at compile time is not a sensible solution (it's not sensible to reject something if you don't even know that it happens). The modern purpose of undefined behavior is to enable these kinds of optimizations anyway, so I can write:

    int *x = some_call();
    ...
    *x = 5;
    ...
    if (x == NULL)
        abort();
The compiler will optimize out the abort(), which is a good optimization since obviously x can't be NULL otherwise *x = 5; would be wrong. You might say that the programmer should be responsible for removing the abort()—but it might be inlined from a different function or something like that. Rejecting the program with an error would seriously piss me off. We want our compiler to be aggressive about optimization under the assumption that if we write clean, standards-conformant code, the compiler will optimize out any bounds checks that it can prove will pass, so we can be free to put bounds checks everywhere and get good performance. That's hard, and it's not obvious that it's hard, and people routinely underestimate how hard it is to write standards-conformant C. There are a lot of alternative ideas in the pipeline but we don't automatically get to blame the compiler vendor, even though sometimes it really has been the compiler's fault.

People get rather riled up about undefined behavior. There is a good subset of people who rather like fast C code (optimizing out paths with UB can make code much faster), and a good subset of people who think C should just be safe and predictable. If you want safe and predictable, C is a tough sell.

Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by kbenson on Friday April 21, 2017 @02:42PM (#14168163)
Arguing the extremes of this as people tend to do about this is not constructive. A compiler flag that warned or errored on branches that were optimized out but the branch included an abort or exit within a certain distance (AST nodes?) of the branch would be useful. Possibly even disallowing optimizing our that conditional that branch if it can be determined it contains any sort of exit or abort before returning control flow.
Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by naasking on Friday April 21, 2017 @03:14PM (#14168466)
> I think there's a misunderstanding here—the undefined behavior occurs at runtime, not at compile time, so rejecting it outright at compile time is not a sensible solution (it's not sensible to reject something if you don't even know that it happens).

Except the compiler clearly knows that it does happen since it elides the code because of it.

> The compiler will optimize out the abort(), which is a good optimization since obviously x can't be NULL otherwise *x = 5; would be wrong.

This isn't an optimization based on undefined behaviour, it's actually a perfectly standard dataflow analysis that even languages with fully defined behaviour would implement.

Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by klodolph on Friday April 21, 2017 @04:02PM (#14168959)
> Except the compiler clearly knows that it does happen since it elides the code because of it.

That's definitely the opposite of what's happening here. The compile believes that it does not happen and therefore removes the code.

Re:Cap'n'Proto remote vuln: pointer overflow check optimized... by kbenson on Friday April 21, 2017 @08:43PM (#14170529)
It's actually an interaction between those two aspects which causes the problem. The compiler makes an assumption that something can't be true because that would mean undefined behavior (and of course the programmer isn't doing something that's undefined), and then a later optimization uses that fact to determine code that tests for that exact undefined behavior could never be run, and removes it. The reasoning by itself is not problematic without it's later use in optimization decisions, and the optimization decisions are not problematic without unfounded reasoning about undefined behavior.

I think that's why people are so caught up on this. Technically, neither action is a problem, but together they obviously caused a problem. Either one in isolation doesn't look all that horrible (even if the undefined behavior reasoning is pushing it).