News for hackers. Stuff that matters.
Re:Moving to a provenance-aware memory object model for C: p... by zik on Sunday April 14, 2019 @05:28AM (#19658366)
Can anyone summarise?
Re:Moving to a provenance-aware memory object model for C: p... by pm215 on Sunday April 14, 2019 @06:06AM (#19658475)
The "n2362+appendix" pdf is only 7 pages if you ignore the appendix which is "changes to the actual C standard to implement the proposal", so it's not that long to read, and it starts with a description of the basic idea. The short summary is that it's trying to formalize an existing idea (which has been discussed by the committee and used by compiler implementations but not really nailed down in the standards text) about when the compiler can assume two pointers don't alias. Intuitively, if you create a pointer by dubious arithmetic means (eg by adding 1 to the address of a local variable) the compiler shouldn't have to assume that that might alias with other local variables, or it would generate terrible code out of paranoia. But you do want common operations where a legitimate pointer is arithmetically manipulated to work. The paper proposes a formalization that accommodates reasonable things but rules some implausible/weird stuff as UB to accommodate both those desires.
Re:Moving to a provenance-aware memory object model for C: p... by simias on Sunday April 14, 2019 @06:59AM (#19658628)
It's interesting work but I really wonder if it's desirable. What real-world problems is this supposed to solve exactly? Making the compiler extra-clever about detecting aliasing seems like a good recipe for having broken code appear to work correctly until it doesn't or getting strange behavior that's a pain to debug. Current aliasing rules are already a significant pain point in the C standard IMO, I personally don't want them to become even more complicated.

The only concrete example given in the paper (having two contiguous objects and referencing the 2nd one through a pointer on the first one) is frankly contrived and I'm not sure how that can be considered reasonable or good code. The only time I could see myself writing code like that is when dealing with memory-mapped hardware registers at fixed addresses which would let me do funky pointer arithmetic, but in this scenario the pointer is usually tagged as volatile and sometimes accompanied by a memory barrier to make sure the compiler does the right thing.

Note that if the programmers really wanted the code given in the paper to work correctly (i.e. changing y through an offset pointer from x) they could do it by adding a memory barrier such as

    asm volatile("": : :"memory");
after the *p = 11;. Is it elegant? Arguably not, but it seems like such a niche use to me that it seems like a fair compromise. It's also not standard C but if you're making such pointer gymnastics chances are you're already making assumptions about your environment that go beyond the standard.

I much prefer Rust's approach of "mutable references never alias" and if the programmer really needs to share a reference they have to manually do the work by wrapping. Of course integrating that into C in a backward-compatible fashion doesn't seem very practical.

Re:Moving to a provenance-aware memory object model for C: p... by pm215 on Sunday April 14, 2019 @10:39AM (#19659647)
The programmers don't want the code in the paper to work correctly, though. What they want is for the C spec to rule it incorrect without also ruling a lot of existing working code to be invalid.

Rust's approach is certainly easier to understand in some ways, but C is not Rust and you can't just change its semantics without considering the existing body of code.

Re:Moving to a provenance-aware memory object model for C: p... by Animats on Sunday April 14, 2019 @03:18PM (#19661140)
C needs an ownership model, but it's almost hopeless to retrofit one. This is a very limited proposal regarding some obscure aliasing problems.

I looked at C and C++ ownership years ago. I had more limited goals than the Rust borrow checker. I was thinking in terms of permissions - read, write, delete, and keep, combined with reference counting.

Keep permission means you can take a copy of a pointer and export it to a larger scope. Like passing in a pointer to an object to be added to a collection. The stored pointer will outlive the return from the function that added it to the collection. It "kept" it.

If you pass a reference to an object without keep permission, the called function can't keep it. So the language does not have to increment the reference count. In fact, you don't even have to pass the reference count, just the raw object pointer. So objects passed by non-keep reference to a function incur no extra overhead. That's a very common case. "Keeping" is relatively rare.

So you get reference count allocation without much overhead.

Workable, but a dead end politically in C. Rust essentially makes all references "non-keep", unless you explicitly use reference counts.

Re:Moving to a provenance-aware memory object model for C: p... by fooker on Sunday April 14, 2019 @07:37AM (#19658788)
>I much prefer Rust's approach of "mutable references never alias"

This rules out a vast majority of fundamental algorithms. Jumping through hoops for getting simple logic working doesn't sound very desirable.

Re:Moving to a provenance-aware memory object model for C: p... by ekidd on Sunday April 14, 2019 @08:40AM (#19659057)
>>I much prefer Rust's approach of "mutable references never alias"

>This rules out a vast majority of fundamental algorithms.

To be more precise, in Rust:

- '&mut T' (a mutable reference to T) can never alias.

- '(asterix) mut T' (a mutable pointer to T) can alias.

The advantage is that you can write most production code with a mix of '&T' and '&mut T', which allows the compiler to assume no mutable aliases anywhere. But if you're building a doubly-linked list, then you can explicitly choose '(asterix) mut T'. (For other options, see https://rust-unofficial.github.io/too-many-lists/.)

In practice, it depends a lot on the code you're writing. I've written tens of thousands of lines of production Rust without needing shared mutability. But if I were writing a traditional game or GUI toolkit, then I might miss shared mutability a lot more.

Re:Moving to a provenance-aware memory object model for C: p... by hedora on Sunday April 14, 2019 @11:18AM (#19659886)
The & semantics sound like fortran’s semantics.

How does the rust compiler know if two references passed in from calling code can alias or not? (I.e., what if I get the type annotations wrong?).

In general, determining if two pointers can alias is equivalent to the halting problem.

The default C way is “assume aliasing unless you can prove otherwise”. This seems safer and like less work to me.

Re:Moving to a provenance-aware memory object model for C: p... by wwright on Sunday April 14, 2019 @11:56AM (#19660120)
Aliasing is checked in the calling code, not the callee. You’re not allow to create a mutable reference if another exists in the same scope. If you use “plain” safe Rust, there’s simply just no way to create alias mutable references unsafely.

(The C default you refer to is at the compiler internals level. The Rust alias prevention we’re talking about is at the language level. C compilers will try to avoid generating code that will behave incorrectly in the presence of aliasing, but do nothing to prevent aliasing and the possible logic and thread safety issues caused by aliasing. Rust makes almost all of these cases disappear for the programmer.)

Re:Moving to a provenance-aware memory object model for C: p... by Ar-Curunir on Sunday April 14, 2019 @01:04PM (#19660443)
The compiler ensures that you can only take one &mut reference at a time. You can have as many immutable `&` references as you want, because you can't change anything through those references (unless you do some unsafe trickery)
Re:Moving to a provenance-aware memory object model for C: p... by cesarb on Sunday April 14, 2019 @03:30PM (#19661194)
> you can only take one &mut reference at a time. You can have as many immutable `&` references as you want

There's another small detail: these two cases are mutually exclusive. You can either have one &mut reference, or as many & references as you want. This means that code receiving a & reference can be sure the object won't change, and that code receiving a &mut reference knows the object can be modified safely since nothing else could be looking at it. (Assuming the object doesn't contain a Cell, the rules are a bit different when a Cell is involved.)

Re:Moving to a provenance-aware memory object model for C: p... by simias on Sunday April 14, 2019 @07:58AM (#19658870)
There's always unsafe. With generics you can always implement these fundamental algorithms once and for all in an unsafe block and then reuse that. No need to reimplement quicksort for every data type.

But I'm not proposing to bring that to C (although it seems that generics are slowly creeping in by way of the macro system), I'm just saying that I think the changes proposed in this paper would make the language more complex without really making it easier to write C code 99% of the time.

Re:Moving to a provenance-aware memory object model for C: p... by FrozenVoid on Sunday April 14, 2019 @10:35AM (#19659625)
> It's interesting work but I really wonder if it's desirable

Aliasing pointers ruing performance of array/pointer access optimizations. https://en.wikipedia.org/wiki/Restrict

Re:Moving to a provenance-aware memory object model for C: p... by simias on Sunday April 14, 2019 @12:44PM (#19660373)
I'm aware of restrict and the problem it solves but that does that say about TFA? Are you proposing that restrict should be deprecated in favor of the mechanism proposed by the parent?

The example in the paper linked is specifically a situation where the compiler does not normally catch the aliasing between two pointers which is the opposite of the problem "restrict" solves (i.e. a situation where the compiler emits code that assumes that two pointers can alias when in fact they do not).

Re:Moving to a provenance-aware memory object model for C: p... by voldacar on Sunday April 14, 2019 @08:20AM (#19658964)
Isn't this exactly what __restrict is for though?
Re:Moving to a provenance-aware memory object model for C: p... by growtofill on Sunday April 14, 2019 @05:06AM (#19658297)
Wow, what a horribly designed website.

Here’s the direct link to the pdf: https://hal.inria.fr/hal-02089889/document

Re:Moving to a provenance-aware memory object model for C: p... by nestorD on Sunday April 14, 2019 @05:27AM (#19658361)
Its a HAL website, the french equivalent of arxiv. Most french research institute have one to publish openly.

https://en.wikipedia.org/wiki/Hyper_Articles_en_Ligne

Re:Moving to a provenance-aware memory object model for C: p... by msla on Sunday April 14, 2019 @10:55PM (#19662892)
It's still a horribly-designed website.
Re:Moving to a provenance-aware memory object model for C: p... by layoutIfNeeded on Sunday April 14, 2019 @07:16AM (#19658698)
Sorry for my ignorance, but my impression was that in the native spaces the consesus is that Rust is the only reasonable choice for any new code. So I don’t understand why people are still working on a C2x standard - shouldn’t the native community work on a roadmap to migrate the existing legacy C/C++ codebases to Rust instead? Am I missing something here?
Re:Moving to a provenance-aware memory object model for C: p... by WoodenChair on Sunday April 14, 2019 @07:43AM (#19658817)
> Sorry for my ignorance, but my impression was that in the native spaces the consesus is that Rust is the only reasonable choice for any new code.

Please tell that to the many millions of C/C++ programmers writing new code every day, who are only vaguely familiar with Rust if at all. If by consensus you mean "consensus amongst the vanguard of Hacker News" then yes. If you mean "consensus" as in the "consensus of the software development industry" then absolutely not.

Re:Moving to a provenance-aware memory object model for C: p... by v_lisivka on Sunday April 14, 2019 @04:14PM (#19661432)
> Please tell that to the many millions of C/C++ programmers writing new code every day, who are only vaguely familiar with Rust if at all.

Just call it "C20", and they will learn it in a week.

Re:Moving to a provenance-aware memory object model for C: p... by adamnemecek on Sunday April 14, 2019 @10:33AM (#19659619)
If you program but don’t post about it on HN, are you really programming?
Re:Moving to a provenance-aware memory object model for C: p... by aduitsis on Sunday April 14, 2019 @10:38AM (#19659641)
Not really, no.

Also, Perl does not exist and is actually an urban legend from the '90s.

Re:Moving to a provenance-aware memory object model for C: p... by detaro on Sunday April 14, 2019 @07:50AM (#19658837)
Such a consensus does in no way exist (even if you assume that there's a consensus on "we need to replace C and C++", which there also isn't, Rust isn't the only "better" native language), and "all new code must be Rust" is mostly a meme by overly enthusiastic enthusiasts. Rust is a tiny sliver of native code written today, not to speak of the orders of magnitude bigger collection of existing code bases and tooling existing outside of it.
Re:Moving to a provenance-aware memory object model for C: p... by jandrewrogers on Sunday April 14, 2019 @11:27AM (#19659929)
You are missing that Rust still can't do many things as well, or as efficiently, or at all as e.g. C++17. It is not suitable for many domains of systems programming. Some safe, simple, idiomatic C++ code is difficult or impossible to express in Rust. Some important safety features in modern C++ don't exist in Rust, so you'd need to remove them from the code base or rewrite the code using a less safe design.

There is room for a new systems programming language but the current solution to the above issues is to make the code "unsafe", slower, and/or more bloated, which offsets its nominal benefits. For many C++ software architectures, memory safety is a non-issue in practice, so that isn't always a compelling feature. The benefits of Rust versus C++ are significantly overstated by its fans, and its current limitations tend to be overlooked. (I can't speak for C, it has been too long since I worked with it at scale.)

Re:Moving to a provenance-aware memory object model for C: p... by c-cube on Sunday April 14, 2019 @11:38AM (#19660000)
What safety feature does rust lack exactly? I know C++ has some more expressive features (like templates on values), but safety? C++ is as unsafe as it gets…
Re:Moving to a provenance-aware memory object model for C: p... by jandrewrogers on Sunday April 14, 2019 @01:36PM (#19660603)
Compile-time code/type generation and verification, which leverages advanced template functionality and constexpr extensively. For applications like database engines with dense, complex data structure interactions, this facility allows you to make fairly deep guarantees about correctness at compile-time that would otherwise require mountains of runtime assertions that as a practical matter programmers won't write with sufficient coverage and may not be tripped in testing in any case. If you make code/design changes, it won't compile if you've introduced one of many common bug types that are otherwise hard to detect when writing code. In my experience, these kinds of bugs are more common than memory safety bugs (which in my experience are a non-issue for modern C++).

This is fairly magical when you use it but it requires a different coding style than some systems programmers are used to. Instead of writing code, you end up writing a lot of meta-code that generates and verifies what actually gets compiled. Recent versions of C++ have invested a lot in making these facilities expressive and reasonable to use in complex software. In my experience this has substantially reduced the code defect rate and simplified testing. Once you've written a system this way, it is very hard to go back.

Not having these facilities would be a huge step backward for writing practical high-assurance/high-reliability code. AFAICT, Rust is still quite primitive in this regard.

Re:Moving to a provenance-aware memory object model for C: p... by 0815test on Sunday April 14, 2019 @02:03PM (#19660745)
> Compile-time code/type generation and verification

Rust does have a procedural macro system which can be used to do these things. Surely actual procedural macros are better than hacking the template system to generate custom code, which is the typical approach in C++.

Re:Moving to a provenance-aware memory object model for C: p... by jandrewrogers on Sunday April 14, 2019 @02:15PM (#19660801)
This is not remotely the same. You are seriously underestimating the capabilities of C++17 in this regard.
Re:Moving to a provenance-aware memory object model for C: p... by JoachimSchipper on Wednesday April 17, 2019 @04:02PM (#19685955)
Do you happen to have a link to a page explaining these techniques? I code in C and dabble in Rust, but this sounds interesting.

(Sorry, I know this is a few days old by now...)

Re:Moving to a provenance-aware memory object model for C: p... by The_rationalist on Sunday April 14, 2019 @11:50AM (#19660084)
On the langage on itself rust lack many expressives features (but "slowly" catch up) that modern c++ has. As for memory safety, I'm not aware of safety features on the Langage on itself but the rust tooling is like third world in comparison. First of all rust has a very incomplete sanitizers support. Ubsan, Msan etc https://github.com/rust-lang/rust/issues/39699 Secondly there is an order of magnitude more static analysis tools that support cpp than rust (but Facebook (a leader in static analysis) is developing a promising one for rust. As for Fuzzers I'm not aware if they're Langage agnostic. Finally there are less (if none?) program provers for rust code.
Re:Moving to a provenance-aware memory object model for C: p... by the_why_of_y on Sunday April 14, 2019 @01:38PM (#19660610)
There may be more static analysis tools available for C++, but the quality of the static analysis built right into Rust's type system blows all of them out of the water; no tool for C++ that I've used (Coverity, Infer, PVS-Studio, ...) can reliably prevent use-after-free or data races.

Due to the type system, safe Rust code has much less need for sanitizers (leaks and lock order checks are useful) or fuzzers (they'll only find infinite loops); granted, you do have a point in that sanitizers are very useful for unsafe Rust code.

There's AFL for Rust: https://github.com/rust-fuzz/afl.rs

[Edit:] With regard to theorem provers, my understanding is that you typically work in some sort of functional language and the correct C/C++ code is generated as a final step at the end (program extraction); certainly Coq and the HOL family work that way.

There's already Rust program extraction for Coq, although I wouldn't be surprised if it's not as mature as C yet: https://github.com/pirapira/coq2rust

In other areas, Rust already has an advantage; compare Cargo with the Babylonian chaos of C++ build sytems, where the question isn't even which one you want to use, but how many different ones are imposed on you by external constraints and how to integrate them without going crazy.

Re:Moving to a provenance-aware memory object model for C: p... by wwright on Sunday April 14, 2019 @12:00PM (#19660133)
Sanitizers and fuzzers are a valid complaint, but static analysis is much less of a problem for Rust vs C++. Most of the issues C++ analyzers detect are prevented by Rust’s design itself, and Rust has an official analyzer in Clippy.

Provers are also a fair complaint, though it’s going to be comparatively super rare that that is an issue in practice :p

Re:Moving to a provenance-aware memory object model for C: p... by pjmlp on Sunday April 14, 2019 @01:22PM (#19660536)
Contracts, static analysis is still in infancy (clippy vs PVS-Studio).

While C's copy-paste compatibility is C++ security story Achilles heel, there is still quite a lot that can be used for writing safer code.

Re:Moving to a provenance-aware memory object model for C: p... by littlestymaar on Monday April 15, 2019 @09:22AM (#19665077)
As far as you only use safe Rust, static analysis is in a way better shape than C++, just because C++ has nothing as powerful as the borrow checker.
Re:Moving to a provenance-aware memory object model for C: p... by pjmlp on Monday April 15, 2019 @04:15PM (#19668509)
Google and Microsoft are working on adding one to clang and Visual C++ static analysers.

https://herbsutter.com/2018/09/20/lifetime-profile-v1-0-post...

Re:Moving to a provenance-aware memory object model for C: p... by littlestymaar on Tuesday April 16, 2019 @02:49AM (#19671635)
And it will probably be incomplete because C++ hasn't be designed with it in mind (How do you avoid shared mutability when everything is mutable by default ?).

Yet, this is good to see C++ trying to catch-up. (And if it's catching up, it means Rust is ahead in terms of static analysis ;)

Re:Moving to a provenance-aware memory object model for C: p... by hedora on Sunday April 14, 2019 @12:02PM (#19660151)
C++ semantics are no less safe than any other language with an “unsafe” mode.

The difference is that C++ makes it convenient to write unsafe code, but that it also makes it possible to write safe systems code at a much higher level of abstraction than any other language I’ve heard of (templates are Turing complete, and modern tooling makes them reasonable to work with).

So, the question is, what does Rust make C++ developers give up that can’t be achieved via C++ style guidelines?

Also, 10 years from now, when the state of the art of systems programming advances, we’ll be able to backport the ideas C++ (just like we do today).

Does Rust have a track record of supporting that?

Re:Moving to a provenance-aware memory object model for C: p... by vvanders on Sunday April 14, 2019 @12:24PM (#19660277)
C++ makes it really easy to corrupt the heap or stack which depending on tooling can take days or weeks to track down. I don't mind trading a bit of flexibility with and optional escape hatch (unsafe).

I'm really skeptical of backporting Rust's wins to C++. The concept of mutability cuts across the language and I don't see how you do it without bifrucating an already complicated language.

Re:Moving to a provenance-aware memory object model for C: p... by UncleEntity on Sunday April 14, 2019 @01:12PM (#19660481)
> The concept of mutability cuts across the language and I don't see how you do it without bifrucating an already complicated language.

How hard would it really be for someone to write a static analyzer for C++ that ensures there is only one non-const reference/pointer to an object active at any given time?

Throw in an object lifetime tracker for allocs and you'd have pretty safe C++ without having to rewrite a billion line of code.

Re:Moving to a provenance-aware memory object model for C: p... by pjmlp on Sunday April 14, 2019 @01:23PM (#19660541)
Microsoft and Google are having a go at it.
Re:Moving to a provenance-aware memory object model for C: p... by c-cube on Sunday April 14, 2019 @03:30PM (#19661196)
It doesn't seem possible to backport pattern matching, sum types, borrow-checker, and the expression based language into C++. This kind of things really needs to be there from the start.

At work we use a pretty big C++ library maintained by absolutely brilliant people, and there still are memory issues and the occasional segfault. It gets tiresome. I'm not convinced anyone can write really safe C++, whereas outside of `unsafe` it's quite easy to write safe rust.

Re:Moving to a provenance-aware memory object model for C: p... by a1369209993 on Sunday April 14, 2019 @03:53PM (#19661322)
> Some safe, simple, idiomatic C++ code is difficult or impossible to express in Rust.

[citation needed]?

I could definitely believe there are such things, but all the examples I've encountered are things like doubly-linked lists where the complainer flatly ignores the existence of the 'unsafe' keyword.

Re:Moving to a provenance-aware memory object model for C: p... by pjmlp on Sunday April 14, 2019 @10:15AM (#19659535)
Rust is very nice as safe systems programming language.

But every sane safety advocate also acknowledges that we aren't going to throw away the millions of lines written in C and C++ tomorrow, and there needs to be a way to keep UNIX derived platforms, while improving their security.

Re:Moving to a provenance-aware memory object model for C: p... by vvanders on Sunday April 14, 2019 @12:17PM (#19660236)
That's why Rust's FFI story of being the C ABI is wonderful. I've used tons of c libraries but still get to take advantage of Rust's advantages in all the new code I write.
Re:Moving to a provenance-aware memory object model for C: p... by raphlinus on Sunday April 14, 2019 @09:35AM (#19659322)
In addition, it is entirely possible that when Rust has a formal semantics, it will use exactly such a provenance-aware memory object model to describe which pointer operations are safe and which lead to UB. People using the safe fragment of Rust won't care about this, but that's certainly not all of Rust.
Re:Moving to a provenance-aware memory object model for C: p... by jfk13 on Sunday April 14, 2019 @07:23AM (#19658731)
Perhaps the "migrate the existing legacy C/C++ codebases to Rust" problem is a lot bigger than you're imagining?
Re:Moving to a provenance-aware memory object model for C: p... by CJefferson on Sunday April 14, 2019 @07:47AM (#19658829)
I love Rust, but there are so many places it isn't feasible (yet). Most microcontroller don't support it for a start.
Re:Moving to a provenance-aware memory object model for C: p... by Ericson2314 on Sunday April 14, 2019 @10:48AM (#19659697)
Eh is think you can make it for anywhere you can use C and clang. Manufacturers write such shit code anywway I don't midnight not using their boilerplate at all.
Re:Moving to a provenance-aware memory object model for C: p... by 0815test on Sunday April 14, 2019 @07:32AM (#19658766)
We're all just waiting for a formal Rust Reference Manual to be published, so that the official ANSI/ISO C and C++ standards can be replaced by a single-line reference to that. Since all practical C/C++ programs will necessarily invoke undefined behavior at some point, doing this is even 100% legal by the previous versions of these standards, even though it means rejecting the older C/C++ syntax altogether; it's really just replacing runtime nasal demons with a far more benign compile-time diagnostic!