News for hackers. Stuff that matters.
Re:Catching integer overflows in C by olliej on Tuesday August 21, 2018 @10:48PM (#17815873)
The issue isn’t what the hardware can do, but rather that the C spec leaves it undefined. Over the last decade have increasingly abused this idea basically making the spec a land mine of UB and then claiming that UB allows them to do whatever they want, even when the programmer intent is obvious.

That said in modern clang and gcc you can (and should) use the overflow builtins which are all essentially:

  Bool __builtin_[operation]_overflow(type a, type b, type* out)
Returning true if the operation overflowed. These produce “ideal” code, e.g

if(overflowoperation(x,y,&out)) ...

Produces the platform branch on overflow instructions.

Re:Catching integer overflows in C by jcranmer on Wednesday August 22, 2018 @12:32AM (#17816322)
> The issue isn’t what the hardware can do, but rather that the C spec leaves it undefined.

Eh, the undefined nature of signed overflow in C is a bit of a red herring. The most common overflow bug in practice is actually 100% well-defined and still 100% wrong: multiplying two numbers to feed into a malloc.

In practice, the undefined signed overflow really only comes into play when you're trying to explicitly check for overflow by doing the operation and then seeing if it overflowed. But, as you mention, compilers give you bultins to do this kind of checked overflow which is quite frankly a much easier way to solve the problem, especially since it's more likely to actually get matched to using hardware flag bits to check for overflow.

Re:Catching integer overflows in C by olliej on Wednesday August 22, 2018 @12:36AM (#17816334)
Yup you are right on that, but the obvious way to check for overflow is UB so gets optimized out. Many security bugs in code people thought was correct. The correct, standard compliant non-UB way to check for the overflow requires a zero check and a division. It’s super non obvious. I like to imagine C committee has made standardized definition for the current builtins but I also have low expectations.
Re:Catching integer overflows in C by xenadu02 on Wednesday August 22, 2018 @12:07AM (#17816219)
> Over the last decade have increasingly abused this idea basically making the spec a land mine of UB and then claiming that UB allows them to do whatever they want, even when the programmer intent is obvious.

That is not an accurate or fair assessment.

However signed integer overflow being undefined behavior is a bit over the top given modern hardware. I'd like to see the C standard adopt a policy of wrapping for signed overflow, then either add new operators or functions for trapping behavior.

Most reasonable hardware has the instructions to trap or branch if add/sub/mul/div overflows or underflows so the checks are very cheap. Unlike the early days, in this day if C made it part of the standard processor manufacturers would simply stop selling CPUs without those features. Same goes for two's-compliment integer representations. They already did the same for 8-bit char/byte; other sizes are no longer supported.

Re:Catching integer overflows in C by olliej on Wednesday August 22, 2018 @12:23AM (#17816285)
There has been an ever increasing creep of undefined behaviour, which has broken existing code - type punning through unions was literally the “correct” mechanism for type coercion, which then became invalid when they established new, questionable, type aliasing rules that rendered it UB. The new type aliasing rules also mean that you can’t implement memcpy/memmove/bzero/memset without hitting UB. That requires hacks in the spec treatment of type aliasing to work around. Hell, even Malloc/free technically is UB due to type aliasing.

Obviously there is signed integer overflow, which has resulted in numerous security flaws for questionable performance gains.

Then there are other fun things, like technically it is UB for bzero to skip padding bytes in a struct (because that is UB), which means bzero(&somestruct, siseof(somestruct)) is not technically guaranteed to clear all of the struct, so I hope there’s not any secrets in left over memory.

Re:Catching integer overflows in C by jcranmer on Wednesday August 22, 2018 @12:34AM (#17816325)
> There has been an ever increasing creep of undefined behaviour, which has broken existing code - type punning through unions was literally the “correct” mechanism for type coercion, which then became invalid when they established new, questionable, type aliasing rules that rendered it UB.

Punning through unions is well-defined behavior in C99 and C11. In C++11 and newer, it's questionable, but it works in practice.

Re:Catching integer overflows in C by saagarjha on Wednesday August 22, 2018 @12:49AM (#17816387)
Punning through a union is undefined in C++11. I'd recommend against doing it in C99 (and newer) as well, even though it has a footnote in the standard saying it's ok, since someone is going to read your code and copy it over to an earlier C project, where it would be undefined. Just use memcpy as intended and let the compiler optimize it out for you.
Re:Catching integer overflows in C by acqq on Wednesday August 22, 2018 @02:15AM (#17816715)
> Punning through a union is undefined in C++11

And that is effectively a bug in that standard, says Linus, and I agree with him.

https://lkml.org/lkml/2018/6/5/769

"I've said this before, and I'll say it again: a standards paper is just so much toilet paper when it conflicts with reality."

Sadly, the compiler maintainers at some points of time managed to push the real "breaking" changes through (for other UB cases). It seems that the "language lawyerism" is "easier" than the expected approach of the responsible engineers.

Re:Catching integer overflows in C by saagarjha on Wednesday August 22, 2018 @02:37AM (#17816802)
I disagree with Linus here. If you're going to go through a complicated set of hoops to make punning work (namely, doing it through a union), what's so hard about using the intended (memcpy) solution? Like, if you really think the standard is that terrible (which isn't what I'm arguing here–I haven't personally made up an opinion on this yet), why not just straight cast and force GCC to adopt a flag that makes your behavior legal?
Re:Catching integer overflows in C by gshrikant on Wednesday August 22, 2018 @04:05AM (#17817136)
I'm curious, AFAICT Linux uses -fno-strict-aliasing with GCC which promises not to enable strict aliasing based optimizations even at higher optimization levels. But they also rely on the GCC blessed behavior of union based type punning which is redundant in that case.

I suppose union based type punning makes the intent a little clearer (vis a vis memcpy) but that's a weak argument at best.

Re:Catching integer overflows in C by saagarjha on Wednesday August 22, 2018 @04:11AM (#17817156)
> I suppose union based type punning makes the intent a little clearer (vis a vis memcpy) but that's a weak argument at best.

How much more explicit can you get than a memcpy?

Re:Catching integer overflows in C by gshrikant on Wednesday August 22, 2018 @04:34AM (#17817246)
I meant the fact that you're actually reinterpreting bits and not really copying them somewhere.
Re:Catching integer overflows in C by AstralStorm on Wednesday August 22, 2018 @03:53AM (#17817093)
The right equivalent is actually punning through char* and enforcing lack of simultaneous aliasing. This is explicitly allowed in C++11.

Memcpy is a cost. Do it only when you actually need a copy.

Re:Catching integer overflows in C by saagarjha on Wednesday August 22, 2018 @04:05AM (#17817135)
All half-decent compilers elide a memcpy when you're using it for this purpose; they'll just generate the assembly you "intended" to create. And while I know casting to a char * is legal for all types, I don't believe casting from a char * is legal unless the memory the pointer references is actually the type you're casting to. I'll have to check on this–or if you have a link to the standard on this I can just look at that.
Re:Catching integer overflows in C by olliej on Wednesday August 22, 2018 @12:39AM (#17816340)
Works in practice resulted in the compilers adding security bugs - literally this article is about overflow checks that worked perfectly fine until the mid-2000s when compiler engineers realized they could abuse UB of overflow for “performance” and then a mountain of previously safe code became insecure.
Re:Catching integer overflows in C by jcranmer on Wednesday August 22, 2018 @12:54AM (#17816410)
Speaking as someone who works on compilers:

Aliasing queries in the compiler basically work on a tiered system. There's a simple aliasing pass that first checks for really obvious alias information (such as, it's the same pointer, it's a bitcast of the pointer, it's a simple arithmetic expression of the same pointer), possibly some more advanced Andersen's-style alias passes, and only finally is there a last-ditch strict-aliasing based pass.

If you take the address of a float and cast that to an integer pointer to get at the bits of the float (and don't let either pointer escape the function), it will work, even with "abusive" compilers that have aggressive optimization turned on, even if you modify the lvalue in the intervening code. It's literally more work to make the compiler not realize that the two pointers alias, and there's no benefit to pessimizing a trivial query, so I would be shocked if any compiler actually miscompiled that code.

Re:Catching integer overflows in C by olliej on Wednesday August 22, 2018 @01:51AM (#17816622)
It “working” isn’t the problem, the problem is when the compiler makes use of the “this is UB” to make formerly correct code incorrect.

The definition of UB would allow a type punned union on the stack for instance to be treated as logically a struct - eg changes to the float value would not impact the integer value. While I agree that would not happen in practice today there is no reason it has to continue to work.

The fact that compilers already make sane decisions regarding some UB - evaluation order of parameters used to be unspecified and gcc at least would change evaluation order depending of optimisation level, but all compilers have realized that just insane and so left to right evaluation. Similarly bzero will initialise padding bytes in a struct even though that is technically UB, so could technically be skipped.

Sorry for run on horror sentences I’m typing on my phone :)