News for hackers. Stuff that matters.
Re:RFC: C++ Buffer Hardening by saagarjha on Thursday October 06, 2022 @01:14AM (#33104621)
> For example, accessing a std::span or a std::vector outside of its bounds would abort the program, and so would accessing an empty std::optional. This can be done while staying Standards-conforming because undefined behavior implies that the library can do whatever it wants, which includes aborting.

sickosyes.jpg

Getting bad operations to abort execution (e.g. by throwing an exception) seems impossible to ship, not in the least due to certain, uh, entities really being against it. But abusing UB to allow people to turn on checks is excellent. This is why we should be very careful about what we choose to standardize-a common request is making signed overflow wrap instead of being UB, which prevents its diagnosis in this way.

Re:RFC: C++ Buffer Hardening by liiisalong on Thursday October 06, 2022 @02:47AM (#33105195)
Turning all UB into IB would be better so a compiler can provide --overflow=crash or --overflow=wrap.
Re:RFC: C++ Buffer Hardening by planede on Thursday October 06, 2022 @04:18AM (#33105835)
UB does not prevent that. Compilers already have flags for those, for example gcc's -ftrapv and -fwrapv .
Re:RFC: C++ Buffer Hardening by liiisalong on Thursday October 06, 2022 @06:02AM (#33106438)
The difference is that implementation defined behavior must be documented and consistent, you get rid of code magically disappearing (null checks after null pointer deref for example) and becoming security problems when you update your compiler.
Re:RFC: C++ Buffer Hardening by kllrnohj on Thursday October 06, 2022 @11:30AM (#33109545)
> you get rid of code magically disappearing (null checks after null pointer deref for example) and becoming security problems when you update your compiler.

You're not going to get that. It would make these proposed safety checks unshipably slow. Languages that have bounds checked array access by default rely on the compiler/JIT being able to recognize in the common case that those branches are dead code & eliminating them. Those "magically disappearing null checks" are in the same category. Mandating that they cannot be optimized away, even though the compiler can "prove" that they are not necessary, isn't going to fly.

Re:RFC: C++ Buffer Hardening by olliej on Thursday October 06, 2022 @04:05PM (#33113350)
First up - bounds checks are simply not expensive, that was true a decade or two ago, it is not now. Similarly null checks are not expensive either. Now the standard C++ "I'll throw an exception now" model for errors is expensive for myriad reasons, and the general mode of development for many projects is to just outright disable exceptions entirely at build time due to the costs that come from the "zero cost" exceptions.

That's why this proposal isn't suggesting throwing an exception - think brk or int3, which is essentially what other bounds checked languages like rust do.

This also isn't mandating that the check can't be thrown away. What it is doing is changing UB to unspecified/implement defined behavior. The difference is important: when something is UB the compiler is free to say "UB is not valid, therefore any preceding branches that would result in UB cannot happen" and then remove them. With unspecified behaviour the compiler only loses the ability to make assumptions based on "this UB path cannot be taken".

C and C++ both have a problem in overuse of undefined vs unspecified behaviour. Take integer overflow: there is no reason that that should be UB, on all hardware it has well defined behavior, and the impact of pretending that this isn't the case has been numerous security exploits over the year. Unspecified behaviour does not (by definition) mean that the spec has any prescriptive behavioral rules beyond "this must be consistent across all sites". For example integer overflow should always be 2's complement or it should always trap, the compiler doesn't get to mix and match, and doesn't get to pretend overflow doesn't happen.

Re:RFC: C++ Buffer Hardening by kllrnohj on Thursday October 06, 2022 @10:09PM (#33116658)
> First up - bounds checks are simply not expensive, that was true a decade or two ago, it is not now. Similarly null checks are not expensive either

They're not expensive because they are optimized away. Branches are still relatively expensive in general. Prediction only goes so far.

> Take integer overflow: there is no reason that that should be UB

Yes there is, it allows for size expansion of the type to match the current architecture. Like promoting a 32 bit index into a 64 bit index for optimal array accesses on a 64 bit CPU. Which you can't do if you're needing to ensure it wraps at 32 bits.

It's not the how it overflows that's the problem, it's the when it overflows that is. Defining it means it has to happen at the exact size of the type, instead of at the optimal size for the usage & registers involved.

> and the impact of pretending that this isn't the case has been numerous security exploits over the year.

And many of those exploits still exist if overflow is defined, because that the overflow happened was the security issue, not the resulting UB failure mode. UB helps here because now it can categorically be called a bug, and you can have things like -ftrapv. That unsigned's wrap is defined and not UB is actually also a source of security bugs - things like the good ol' `T* data = malloc(number * sizeof(T));` overflow bugs. There's no UB there, so you won't get any warnings or UBsan traps. Nope, that's a well-defined security bug. If only unsigned wrapping had been UB that could have been avoided...

But if you want code that is hyper consistent and behaves absolutely identically everywhere regardless of the hardware it's running on (which things like crypto code does actually want), then C/C++ is just the wrong language for you. By a massive, massive amount. Tightening up the UB and pushing some stuff to IB doesn't suddenly make it a secure virtual machine, after all. You're still better off with a different language for that. One where consistency of execution is the priority, abstracting away any and all hardware differences. But that's never been C or C++'s goal.

C/C++ do have problems with UB. But int overflow isn't actually an example of it. And unsigned's overflow being defined is also really a mistake, it should have been UB.

Re:RFC: C++ Buffer Hardening by GoblinSlayer on Thursday October 06, 2022 @01:40PM (#33111386)
Yep, behavior is consistent with the implementation code.
Re:RFC: C++ Buffer Hardening by gpderetta on Thursday October 06, 2022 @05:47AM (#33106348)
in fact it is exactly UB that allows that. If, say, the standard specified wrap behaviour, a conforming compiler wouldn't be allowed to trap.
Re:RFC: C++ Buffer Hardening by planede on Thursday October 06, 2022 @05:51AM (#33106365)
You are right, but OP asserted that IB (implementation defined behavior) would be needed for this. IB would also allow this, but it is not necessary.

All of undefined/unspecified/implementation-defined allows this.

edit:

Whether unspecified or implementation-defined would allow trap depends on the exact wording. If the wording is along the lines of "[on overflow] the value of the expression is unspecified/implementation-defined" then trap is not allowed. But it can be also "[on overflow] the behavior of the program is implementation-defined", which would also allow trap. Point being, unspecified/implementation-defined also have a defined scope, while undefined behavior always just spoils the whole program.

I'm fine with the UB overflow though.

Re:RFC: C++ Buffer Hardening by saagarjha on Thursday October 06, 2022 @06:01AM (#33106434)
> the behavior of the program is implementation-defined

You're basically defining undefined behavior. The way to do this "properly" would be "the program is allowed to do x, y, z, or also trap"–there's really not much else you can do.

Re:RFC: C++ Buffer Hardening by planede on Thursday October 06, 2022 @06:28AM (#33106577)
The difference here is that the behavior would need to be documented.

That's why I didn't bother with "the behavior of the program is unspecifed", because that really is just undefined behavior.

But yeah, actually listing allowed behaviors and stating that the actual behavior out of them is unspecified/implementation-defined is more practical.

One example is conversion to signed integer in C17:

> ...either the result is implementation-defined or an implementation-defined signal is raised.

https://cigix.me/c17#6.3.1.3.p3

Re:RFC: C++ Buffer Hardening by gpderetta on Thursday October 06, 2022 @06:10AM (#33106479)
Well, it is not that easy. Specifying a subset of allowed outcomes might still prevent future evolution.

For example bound checking, while fairly cheap for serial code, makes it very hard to vectorize code effectively. Then again so does IEEE FP math and that's why we have -ffast-math...

In the end, it is not impossible, it is just tradeoffs.

I'm generally on the opinion that there should be as little UB as possible, but probably this is better done by compilers that can push the limit of static and dynamic checking technology instead of the standard mandating it.

Re:RFC: C++ Buffer Hardening by saagarjha on Thursday October 06, 2022 @06:13AM (#33106493)
Right, that’s my point at the top of the thread :) I’m just saying if you don’t end up with something that looks like what I just mentioned you’re really not specifying anything, so you might as well go for undefined behavior. This is fine.
Re:RFC: C++ Buffer Hardening by gpderetta on Thursday October 06, 2022 @07:42AM (#33106983)
Yes, I think we are in complete agreement :)
Re:RFC: C++ Buffer Hardening by olliej on Thursday October 06, 2022 @04:05AM (#33105730)
IB?
Re:RFC: C++ Buffer Hardening by flohofwoe on Thursday October 06, 2022 @04:33AM (#33105943)
Implementation defined behaviour.
Re:RFC: C++ Buffer Hardening by matthews2 on Thursday October 06, 2022 @04:34AM (#33105950)
Implementation defined behaviour
Re:RFC: C++ Buffer Hardening by olliej on Thursday October 06, 2022 @04:52AM (#33106066)
Ah yeah, "unspecified behaviour" is my known phrase, but I can see it would be confusing to be arguing about whether something should be UB or UB :D
Re:RFC: C++ Buffer Hardening by planede on Thursday October 06, 2022 @05:17AM (#33106214)
Unspecified behavior and implementation defined behavior are distinct in theory. The latter requires the behavior to be documented by the implementer, while the former does not.

In practice I dare you to find compiler documentation for each implementation defined behavior.

https://timsong-cpp.github.io/cppwp/n4868/intro.defs#defns.u...

https://timsong-cpp.github.io/cppwp/n4868/intro.defs#defns.i...

edit:

C wording:

https://cigix.me/c17#J.1

https://cigix.me/c17#J.3

Re:RFC: C++ Buffer Hardening by gpderetta on Thursday October 06, 2022 @05:50AM (#33106362)
> In practice I dare you to find compiler documentation for each implementation defined behavior.

For gcc there is this:

https://gcc.gnu.org/onlinedocs/gcc/C-Implementation.html

No idea how complete it is. Also a lot of stuff is architecture/platform specific, not compiler specific, so you won't find it in the general compiler docs but you have to look at the psABI.

Re:RFC: C++ Buffer Hardening by olliej on Thursday October 06, 2022 @02:10AM (#33104947)
Making out of bounds access on explicitly bound types is an easy memory safety win.

The fact the UB is still being used as an excuse to clearly undermine reasonably expected behaviour remains a BS trope, and WG21's continued acceptance of new features and specifications that say perfectly specifiable behaviour is UB instead of unspecified is a continuing source of avoidable security bugs.

It is inexcusable for new features to be added to C++ that have UB in any case other than the undefinable (invalid memory access, etc). Removing the copious unnecessary UB from the existing specs should be a higher priority that many of the new features being proposed.

Re:RFC: C++ Buffer Hardening by saagarjha on Thursday October 06, 2022 @02:50AM (#33105222)
Much that is UB deserves to be defined. And some that is defined deserves to be UB. Can you make it illegal? Then do not be too eager to deal out behaviors in standardization.

I don't actually oppose, in the long run, slowly picking some behaviors and changing them to be standardized. But I do vehemently oppose many of the suggestions that are brought up. The kinds of things that should be moved out of being undefined are things like "you named your function isfoo" and not "you passed NULL to memset".

Re:RFC: C++ Buffer Hardening by olliej on Thursday October 06, 2022 @04:05AM (#33105726)
> And some that is defined deserves to be UB.

Can you provide an example of currently specified behaviour that should be UB?

To be very clear I am using specification language here. "Unspecified behaviour" is very much not "undefined behaviour".

Re:RFC: C++ Buffer Hardening by flohofwoe on Thursday October 06, 2022 @04:35AM (#33105956)
It's an LoTR joke: "Many that live deserve death. And some that die deserve life. Can you give it to them? Then do not be too eager to deal out death in judgement"
Re:RFC: C++ Buffer Hardening by mgaunard on Thursday October 06, 2022 @04:41AM (#33105994)
Undefined behaviour is great. I for one will continue to vote in its favour.

C++ does not add runtime overhead to validate that programs are well-formed.

Re:RFC: C++ Buffer Hardening by olliej on Thursday October 06, 2022 @05:13AM (#33106191)
Can you provide examples of UB that could be unspecified behaviour, where UB does anything other than cause harm?
Re:RFC: C++ Buffer Hardening by mgaunard on Thursday October 06, 2022 @09:33AM (#33107960)
Dereferencing a pointer to memory that doesn't point to a live valid object is undefined behaviour. You couldn't possibly specify what this does without instrumenting every single access to memory, like asan or valgrind does.
Re:RFC: C++ Buffer Hardening by olliej on Thursday October 06, 2022 @01:30PM (#33111250)
As I said, behaviour that could be unspecified rather than undefined.

This is an example of something that cannot be anything other than UB.

Re:RFC: C++ Buffer Hardening by kllrnohj on Thursday October 06, 2022 @11:33AM (#33109566)
Re:RFC: C++ Buffer Hardening by staticassertion on Thursday October 06, 2022 @05:02AM (#33106131)
> C++ does not add runtime overhead to validate that programs are well-formed.

You have succinctly described the problem.

Re:RFC: C++ Buffer Hardening by mgaunard on Saturday October 08, 2022 @04:50AM (#33130830)
That's not a problem, that's a core policy of that language's design.

If you want a language that is slow, there are plenty of options.

Re:RFC: C++ Buffer Hardening by avgcorrection on Thursday October 06, 2022 @05:22PM (#33114263)
Compilers optimizing based on UB: normal

Acting in a crash-predictable way on UB: abuse

A strange world this be.