News for hackers. Stuff that matters.
Re:A Provenance-aware Memory Object Model for C [pdf] by 1ris on Saturday June 19, 2021 @07:31AM (#27559635)
Disregard this. I was wrong.
Re:A Provenance-aware Memory Object Model for C [pdf] by Blaisorblade0 on Saturday June 19, 2021 @08:44AM (#27559985)
First, this change is reducing undefined behavior* (EDIT: see below), and that undefined behavior (most casts between pointers and integers) did _not_ work reliably in practice. And in fact, the semantics they chose is rather sensible.

But yes, some code with UB which works for now might break. But that’s true whenever you upgrade your C compiler or use optimizations, so for correctness you shouldn’t do either! People working on certified embedded software have known that for long.

The real problem is that C users have been lied to. They’ve been taught the lie that C pointers are just addresses, that memory is an array of bytes, that relying on this lie is robust, and that this combination is fast. But this combination is impossible.

EDIT: officially some behavior was implementation-defined, but implementations made it undefined anyway — either officially or via bugs; links in subthread.

Re:A Provenance-aware Memory Object Model for C [pdf] by 1ris on Saturday June 19, 2021 @08:53AM (#27560045)
Then i think i was wrong. For me this looks like it _added_ a whole lot of UB and made the rules of object provenance way more complex.

>The real problem is that C users have been lied to.

I think the real problem is that the committee is creating a language that the users neither need nor understand. The users want a language where pointers are, in fact, just address.

Re:A Provenance-aware Memory Object Model for C [pdf] by Blaisorblade0 on Saturday June 19, 2021 @09:13AM (#27560182)
Why don’t they just disable optimizations then? I think that’s because they also want performance, even when it requires pointers to not be just addresses.

And pointers weren’t addresses before either. My favorite example of “pointers aren’t addresses” is this kind of code:

int a[…]; int b[…]; /* … */ a[i] = 0; b[j] = 1; return a[i];

Ignoring this discussion for a moment, wouldn’t you expect `return a[i];` to optimize to `return 0;` here? After all, `a` and `b` are disjoint arrays! I’m sure many would be disappointed if the compiler did not do that optimization, right?

Re:A Provenance-aware Memory Object Model for C [pdf] by 1ris on Sunday June 20, 2021 @05:36AM (#27568505)
If i wanted the compiler to generate code for `return 0;` I'd write it that way. If i wanted to use c as my macro assembler i very much expect the compiler to _not_ make this transformation.

If I wanted language with clever optimisations I'd go for something way higher level like lisp, haskell. Most optimisations are often only useful to make sure you can program generically without a headache. Optimizing code generated by macros or monomorphisation/type specialisation/template instantiation/however you call it is really useful. Is it worth it otherwise? For me the difference between gcc -O1 and -O3 suggest this is not worth it in the domain where c is typically used.

Re:A Provenance-aware Memory Object Model for C [pdf] by SAI_Peregrinus on Saturday June 19, 2021 @04:06PM (#27563957)
Optimizations don't really matter here. Optimization can't cause non-conforming code to be emitted (for a non-buggy compiler). A non-optimizing compiler could emit exactly the same code. If they don't, that's just inefficient code gen by the non-optimizing compiler.
Re:A Provenance-aware Memory Object Model for C [pdf] by astrange on Saturday June 19, 2021 @08:31AM (#27559910)
The point of this is to reduce miscompilations caused by pointer-int-pointer casts, use-after-free crashes, etc.
Re:A Provenance-aware Memory Object Model for C [pdf] by 1ris on Saturday June 19, 2021 @08:47AM (#27560002)
Maybe i missunderstood and overracted. I me it looked (and looks) like it introduces a whole lot of new UB and making pointer even more abstract.
Re:A Provenance-aware Memory Object Model for C [pdf] by astrange on Saturday June 19, 2021 @09:12AM (#27560172)
If it helps, anything that introduces UB for optimizations (like forbidding assuming that 'int y,x;' implies an order in memory) is also good for security.

Mainly because it lets you use systems with type-safe pointers (some acronyms are PAC, BTI, CHERI) - the regular "concrete semantics" aka "whatever current pointers let you get away with" has a lot of issues! But also because compiling with extra security is not popular if it causes performance regressions.

Re:A Provenance-aware Memory Object Model for C [pdf] by temac on Saturday June 19, 2021 @10:16AM (#27560615)
> anything that introduces UB for optimizations (like forbidding assuming that 'int y,x;' implies an order in memory) is also good for security.

[...]

> But also because compiling with extra security is not popular if it causes performance regressions.

Yes. So while UB could theoretically be (in some cases) good for security, in practice it is absolutely not, because no tooling exists to introduce runtime checks light enough to be used in production. So UBs can and do reveal and/or amplify latent bugs, or even introduce new ones if new UB are introduced in new specifications.

Re:A Provenance-aware Memory Object Model for C [pdf] by astrange on Saturday June 19, 2021 @10:30AM (#27560709)
PAC/BTI are definitely used in production, UBSan is fast enough to ship in production if you want, and there's more tools being used that I can't talk about here.
Re:A Provenance-aware Memory Object Model for C [pdf] by rualca on Saturday June 19, 2021 @07:48AM (#27559708)
> I don't think the world has a need for faster c code. But is in dire need for less broken software. This is trying to do the opposite.

And that's ok. There are tools for that at the reach of those who place a premium on those requirements.

For those who favour interoperability, simpler mental models, efficiency, and more importantly a tool with a proved track record, there's C.

And there are plenty of reasons why C is the de facto standard in spite of all alternatives.

Re:A Provenance-aware Memory Object Model for C [pdf] by dooglius on Saturday June 19, 2021 @07:58AM (#27559753)
Are you genuinely claiming that this makes for a SIMPLER mental model (as compared to the implicit "concrete" model)? I don't see how you can justify that...
Re:A Provenance-aware Memory Object Model for C [pdf] by keithalewis on Saturday June 19, 2021 @08:39AM (#27559952)
If you are trying to mentally model how the keys you are pressing will be turned into the machine instructions that will ultimately get executed, yes. But that is no excuse for not looking at assembly.
Re:A Provenance-aware Memory Object Model for C [pdf] by Blaisorblade0 on Saturday June 19, 2021 @08:48AM (#27560008)
But you can’t compare to the concrete model, because it’s not the status quo. That ship already sailed with C89 and strict aliasing. Ptr2int casts were never legal, and support was inconsistent.

For instance, GCC sometimes tracks pointer provenance through integers; this proposed change will make that clearly illegal.

Re:A Provenance-aware Memory Object Model for C [pdf] by dooglius on Saturday June 19, 2021 @09:06AM (#27560142)
The concrete model was raised explicitly as a possible option, why can't I compare to that? TFA says that the status quo is ambiguous.

>Ptr2int casts were never legal

Citation on this? The existence of uintptr_t seems to say otherwise.

>That ship already sailed

In principle, I see no reason preventing future versions of the standard from defining previously-undefined behavior. That said, you are probably right in a realpolitik sense.

Re:A Provenance-aware Memory Object Model for C [pdf] by Blaisorblade0 on Saturday June 19, 2021 @09:22AM (#27560235)
Should have been more careful — in C and C++ they were implementation-defined. But both theory and practice did not mandate a concrete model — GCC already violates that and implements a mishmash of PVI and PNVI.

From N2176, Sec. 6.3.2.3, p. 5-6: > An integer may be converted to any pointer type. Except as previously specified, the result is implementation-defined, might not be correctly aligned, might not point to an entity of the referenced type, and might be a trap representation.

> Any pointer type may be converted to an integer type. Except as previously specified, the result is implementation-defined. If the result cannot be represented in the integer type, the behavior is undefined. The result need not be in the range of values of any integer type.

AFAICT, that doesn’t even guarantee that a roundtrip is the identity — which C++ mandates and is still the case: http://eel.is/c++draft/expr.reinterpret.cast#5

EDIT: GCC does have those restrictions in place already: https://gcc.gnu.org/onlinedocs/gcc-11.1.0/gcc/Arrays-and-poi...

and since a while (earliest relevant docs I could find easily): https://gcc.gnu.org/onlinedocs/gcc-3.1.1/gcc/Arrays-and-poin...

EDIT2: LLVM doesn’t document this “implementation-defined behavior” (or most others), even if docs are explicitly required by the standard; but it does use pointer provenance in practice already, but not in a way that’s easy to explain.

Re:A Provenance-aware Memory Object Model for C [pdf] by pjmlp on Saturday June 19, 2021 @08:49AM (#27560017)
UNIX and C being available as free beer since their inception (actually 6th edition) kind of outnumbers all the other possible reasons.
Re:A Provenance-aware Memory Object Model for C [pdf] by rualca on Saturday June 19, 2021 @01:28PM (#27562430)
> UNIX and C being available as free beer since their inception (actually 6th edition) kind of outnumbers all the other possible reasons.

That would only explain why C featured as one of the choices on the table.

But from that point on, people like you and me made conscious decisions based on their requirements and preferences, and C frequently was everyone's top choice.

Re:A Provenance-aware Memory Object Model for C [pdf] by pjmlp on Saturday June 19, 2021 @01:45PM (#27562578)
Just like JavaScript is frequently everyone's top choice for the browser, and PHP is frequently everyone's top choice for making their websites dynamic in most ISPs.

Quality is surely not the common factor to all those top choices, rather being the platform owners, with all that it entails to the surrounding ecosystem.

Re:A Provenance-aware Memory Object Model for C [pdf] by rualca on Saturday June 19, 2021 @02:58PM (#27563310)
> Just like JavaScript is frequently everyone's top choice for the browser

No, not really, and frankly this feels like a disingenuous comparison done in bad faith. I'm sure you know that, unlike C, javascript is pretty much the only option on the table if your goal is to produce anything remotely resembling production code. Also, as you certainly are aware, that's far from the case with C. In fact, you should certainly be aware that one of the most popular compiler toolkits there is, and one which has been the de facto standard for a few decades now, offers C only as a front-end, on par with languages such as Ada, which already was around for close to a decade before C's first international standard was published.

Thus, you'll need to come up with an explanation of why people repeatedly pick C over well established languages explicitly designed to meet he requirements you favour, and one which is already around for four decades, and is already standard and widely available and enjoys some adoption in some fields. But in spite of it everyone still picks C over it.

Re:A Provenance-aware Memory Object Model for C [pdf] by pjmlp on Sunday June 20, 2021 @03:18AM (#27567886)
C is the only option in UNIX/POSIX platforms like 99% of the time.

The other being C++, which happens to be for better or worse, almost copy-paste compatible with C89, also also a UNIX child born a couple of offices down the corridor.

Interesting that you pick Ada, given how outrageous the compilers used to cost, versus the freebie C compiler that already came on the box, and cleverly washout the complaints on the FOOS community that won't touch GNAT due to licensing FUD.

Who is on bad faith here?

Re:A Provenance-aware Memory Object Model for C [pdf] by johnthescott on Saturday June 19, 2021 @04:41PM (#27564279)
bravo.
Re:A Provenance-aware Memory Object Model for C [pdf] by 1ris on Saturday June 19, 2021 @08:31AM (#27559905)
I don't get what you are trying to tell me. People use C. Yes, it get it. I actually argued infafour of that. Embedded development, device drivers and kernels.

None of these need speed. They need reliability and maintainability. The standards committee is prying the language out of the hands of these people. Because they need, using your words, "interoperability, simpler mental models," which is sabotaged by efforts like this one.