News for hackers. Stuff that matters.
Re:Signed Integers Are Two’s Complement by raverbashing on Thursday May 31, 2018 @02:40AM (#17194403)
> Naïve overflow checks, which are often security-critical, often get eliminated by compilers. This leads to exploitable code when the intent was clearly not to and the code, while naïve, was correctly performing security checks for two’s complement integers.

This is the most critical aspect. We have enough trouble already without the compiler actually fighting against security because this would fail in a machine from the 70s

Re:Signed Integers Are Two’s Complement by virgilp on Thursday May 31, 2018 @10:25AM (#17196521)
Why are you assuming "machine from the 70s"? I know modern processors (DSPs) that need to saturate on integer arithmetic in order to maintain correctness. If you want your naive overflow checks to work on your x86 project, why not just use a compiler option like -fwrapv?

IMO, this is just plain ignorance - people arguing against the standard, while believing only their favorite platform is significant. C code is still big in embedded, you can't just trash the standard like that.

Re:Signed Integers Are Two’s Complement by raverbashing on Thursday May 31, 2018 @11:19AM (#17196942)
I've used DSPs and embedded systems, those compilers have sufficient quirks already and usually don't follow the standard to the letter. I'm not worried about "signal" values, I'm worried about pointer arithmetic values.

x86 SIMD is saturating as well

> If you want your naive overflow checks to work on your x86 project, why not just use a compiler option like -fwrapv?

Fair enough. Or people can stop pretending that UB is just an excuse to throw your hands in the air and do whatever they want with the code. Including null checks.

Because when things blow up it's on the major platforms.

Re:Signed Integers Are Two’s Complement by sanxiyn on Thursday May 31, 2018 @03:21AM (#17194570)
This got rejected in the next revision of this proposal. Naive overflow checks are still undefined.
Re:Signed Integers Are Two’s Complement by dzdt on Thursday May 31, 2018 @08:46AM (#17195852)
If a signed operation would naturally produce a value that is not within the range of the result type, the behavior is undefined. The author had hoped to make this well-defined as wrapping (the operations produce the same value bits as for the corresponding unsigned type), but WG21 had strong resistance against this.

Who are the standards committee people having strong resistance against this, and what in the world is their argument?

Re:Signed Integers Are Two’s Complement by kazinator on Thursday May 31, 2018 @08:59AM (#17195921)
Undefined doesn't mean incorrect; it's just the absence of a requirement. A compiler writer can add requirements locally (like "signed integer overflows have wrapping behavior") that are missing in the standard.

The standard is not a suitable babysitting tool for GCC maintainers, which is what I suspect is the motivation here.

Re:Signed Integers Are Two’s Complement by yoklov on Thursday May 31, 2018 @09:26AM (#17196090)
This isn’t entirely untrue in theory, but in practice it is. Compilers optimize under the assumption that undefined behavior never occurs, and the standard is written with this assumption. Any conforming compiler is free to break code that relies on this.

Typically, behavior that reasonably may vary from compiler/machine is considered implementation defined, and not undefined.

Re:Signed Integers Are Two’s Complement by kazinator on Thursday May 31, 2018 @09:51AM (#17196272)
> Any conforming compiler is free to break code that relies on this.

Ah, but, for example, __attribute__((packed)) is undefined behavior; is the compiler free to break that?

This "free to break" is a juvenile fiction based on the idea that the only document that applies is ISO C; there is no other contract or promise between user and implementor.

Re:Signed Integers Are Two’s Complement by sanxiyn on Thursday May 31, 2018 @10:03AM (#17196360)
GCC documentation is another such contract, which provides -fwrapv. If you want -fwrapv, use -fwrapv.
Re:Signed Integers Are Two’s Complement by sanxiyn on Thursday May 31, 2018 @10:01AM (#17196348)
I am not sure why people pick GCC in particular for this issue. LLVM does exactly the same.
Re:Signed Integers Are Two’s Complement by gpvos on Thursday May 31, 2018 @04:37AM (#17194886)
Okay, then the proposal is already mostly useless.
Re:Signed Integers Are Two’s Complement by fulafel on Thursday May 31, 2018 @03:47AM (#17194681)
Isn't the quoted part 180 degrees wrong? Such code was not "correctly performing security checks", since it was undefined behaviour - two's complement or not. Which was the whole problem.
Re:Signed Integers Are Two’s Complement by loup-vaillant on Thursday May 31, 2018 @08:04AM (#17195674)
Adding "assuming the compiler is not insane and uses -fwrap" would be a bit cumbersome.

The fact remains that the checks looks correct to anyone familiar with 2's complement, yet unfamiliar with the intricacies of the C and C++ standard, which are not low level, contrary to what they were being told at school.

The resistance of the committee about this very issue, as shown by revision 2 of this proposal¹, is enough to make me hope C and C++ will go the way of COBOL.

That said, 2's complement guarantees that converting everything to unsigned before performing an operation, then converting back afterwards, is guaranteed to produce the same results as -fwrap (except for division and modulo). Source to source transformation tools may help us bypass undefined behaviour without resorting to non-standard compiler flags, or implementation defined behaviour.

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p090...

Re:Signed Integers Are Two’s Complement by iainmerrick on Thursday May 31, 2018 @04:29AM (#17194859)
The checks were not correct for all possible representations. If the only representation were two’s complement, there could be fewer areas of undefined behavior, so simple and straightforward security checks would be much more likely to be correct.

(Unfortunately it sounds like this proposal has been revised to leave overflow behavior as undefined. That’s a biggie. Oh well)

Re:Signed Integers Are Two’s Complement by mannykannot on Thursday May 31, 2018 @09:02AM (#17195929)
> The checks were not correct for all possible representations.

This raises the question of whether the optimization was valid, given the target architecture. I am assuming that, technically speaking, it was, because that is what the standard allowed, but that view leaves unexamined the question of whether it is contrary to the purposes for having and using the C language in the first place.

I can imagine an argument pointing out that this optimization is applied at an abstract level of representation prior to code generation, and that it would be a violation of modularity to take into account the target architecture at that point. This, however, would be a point about the compiler architecture, which I think should, where practical, yield to concerns about the overall purpose and use of the compiler, where the principle of 'no (or minimal) surprises' is important.

Re:Signed Integers Are Two’s Complement by gpvos on Thursday May 31, 2018 @04:36AM (#17194881)
Don't leave out the last part: ...was correctly performing security checks for two’s complement integers.

More pedantically, they could have written: ...was correctly performing security checks if the standard had been limited to two’s complement integers. But that was clearly the intent.

Re:Signed Integers Are Two’s Complement by fulafel on Thursday May 31, 2018 @11:59AM (#17197318)
But the revised version is limited to twos complement and is still undefined, so such a check is still incorrect.
Re:Signed Integers Are Two’s Complement by gpvos on Friday June 01, 2018 @04:54AM (#17203674)
Okay, you may be right; I haven't looked through all diffs in detail (nor do I understand all intricacies of the standard).
Re:Signed Integers Are Two’s Complement by mannykannot on Thursday May 31, 2018 @08:36AM (#17195803)
This is a good example of why the aphorism 'technically right is the best kind of right' is problematical (of course, those who like it will point out that I am not technically correct...)

Regardless, the fact that someone wrote code that specified undefined behavior and got what they asked for instead of what they wrote is not the whole problem. Unless this outcome is what the standards committee wanted (in which case we have a different problem), then it is very reasonable to ask the question of whether we are making staying within defined behavior unduly difficult through rules that refuse to treat nearly hypothetical circumstances as special cases.

Re:Signed Integers Are Two’s Complement by JdeBP on Thursday May 31, 2018 @09:29AM (#17196114)
The whole problem is in fact this.

* The language standards came from a time where there was no standard (de facto, that is) for signed integer arithmetic across instruction architectures. Bear in mind that many people involved in standardization (rightly) want to standardize what is in actual practice in the world. If the world hasn't settled on one thing, it is difficult to standardize. (It's why the system administration parts of Unix were not addressed by IEEE 1003.1, for example. There were a whole lot of significantly different ways in which system administration was done.)

* Programmers were coding "knowing" that 2s-complement arithmetic led to certain tricks for detecting overflow and other sorts of bit twiddling (https://news.ycombinator.com/item?id=17044546); "knowing" that their processor architectures were 2s-complement; and "knowing" that compilers naively just translated straight to the arithmetic machine instructions of the target architecture.

* Compiler implementors were writing compilers knowing that programmers did not in fact have these guarantees, and implementing their optimizers as if the target processor architectures were not 2s-complement (in particular, as if integers had infinite bits); even when the actual machine code generation parts of their compilers were designed with the knowledge that the target processor architecture was 2s-complement.

The whole problem is that this is a mess that does not hang together.

There are several ways out of it. One is to make Sean Eron Anderson's life a living hell (https://graphics.stanford.edu/~seander/bithacks.html), and attempt to stamp out every piece of samizdat doco and programmer folklore that circulates these tricks, or at least make every one of them carry a lengthy "health warning" that the world is not, in fact, guaranteed to provide 2s-complement arithmetic to programmers. Another is to give in and say that the heretofore unwarranted assumptions by the programmers are now in fact supported, and that the compiler implementors have to change their now invalid compiler designs.

A third is to do part of each, by accepting and legitimizing the programmer folklore to an extent, but realizing that programmers often "know" quite the opposite case and assume that they are not using 2s-complement arithmetic. Where one programmer can be surprised to find that (x + 1) > (x) is always true because on the 2s-complement architecture that xe expects it isn't; another programmer can be surprised to find that ((x * 2) / 2) == (x) is not always true because in elementary school arithmetic multiplication by 2 is the inverse of division by 2, and be further surprised that (say) some deep nesting of macros that results in such things doesn't reduce to a no-op.

Re:Signed Integers Are Two’s Complement by Bromskloss on Thursday May 31, 2018 @11:46AM (#17197172)
By all means, make the change for the new version of the language. In the meantime, though, why are people assuming two's complement?!
Re:Signed Integers Are Two’s Complement by gpvos on Friday June 01, 2018 @05:00AM (#17203699)
General knowledge. Have you read the entire standard? I haven't, but I know my machine uses two's complement internally, and C and C++ are generally supposed to be close to the bare metal.