News for hackers. Stuff that matters.
Re:The struggle with Rust by Manishearth on Tuesday January 24, 2017 @01:46PM (#13474237)
Rust contests the claim that you actually need to do pointer arithmetic and stuff often in systems languages. You need to do it a bit to implement your abstractions, maybe, but it is not the first tool you should need to reach for.

The code in the blog post looks very much like C-translated-to-rust. Right now, systems programming mostly does look like C/C++ because those are the only two mainstream languages that work here (D, too, but D is like C++. There are also other languages). But systems programming doesn't have to look the same or use the same primitives.

Do you really need to use malloc to allocate memory in a systems language? Do you really need to use pointer arithmetic to deal with things in a systems language? Sometimes, but not necessarily often. Even the Rust operating systems have managed to minimize the amount of unsafe/C-like code.

Rust does expose this functionality, but if it's not something it expects you to use often it makes sense that it might not always be straightforward to use.

Re:The struggle with Rust by pjmlp on Tuesday January 24, 2017 @02:18PM (#13474545)
> Do you really need to use malloc to allocate memory in a systems language? Do you really need to use pointer arithmetic to deal with things in a systems language?

To add to this point, my answer would be a big NO, if one would look to the C alternatives that sadly became niches.

Sadly the adoption of C has lead to younger generations thinking that those are the only valid paths in system languages.

Thankfully new languages, including Rust, are changing that.

Re:The struggle with Rust by AstralStorm on Tuesday January 24, 2017 @03:42PM (#13475274)
However, typed and separate memory areas with various allowed kinds of concurrent access are needed. Even simple RAM has multiple modes, plain, atomic of multiple kinds, barriers... On top of that, you get devices that handle access only with delay inbetween, can never be accessed concurrently (volatile), have alignment requirements and more. Rust memory model is too weak to enforce safety in such cases while not sacrificing performance.

It is written from "I have one CPU" side. Even C is far beyond that with things like OpenCL...

Re:The struggle with Rust by dbaupp on Tuesday January 24, 2017 @06:05PM (#13476641)
Rust gets all your favourite kinds of atomics and barriers, exactly like C. Other things are indeed hard to get safe, which is why Rust has `unsafe`: it is possible to use them, marking out the dangerous regions precisely, rather than the whole program being `unsafe` just because a few places need niche features.

In any case, Rust is definitely not written with "I have one CPU" in mind. A slogan that is sometimes used for Rust is "safe. fast. concurrent.", and it's one of the headline features on the front page: http://rust-lang.org/ . Rust's abstractions allows writing extremely high performance libraries like rayon, which, had better performance than even highly optimised C libraries like Cilk, in one benchmark I heard about. Building out that parallelism ecosystem takes time, so it does not seem particularly surprising that there are more (unsafe!) libraries for the 40-year-old language, than (safe!) libraries in the less-than-2-year-old one.

Re:The struggle with Rust by AstralStorm on Wednesday January 25, 2017 @03:20AM (#13479691)
No, by one CPU i meant not a single core. I meant not having to manage different memory models and available access patterns. Or different ways of handling race conditions and alignments.

You get to use unsafe almost exclusively in the internals of such access where it shouldn't be necessary if there are ways to specify more properties than just binary "safe" and "unsafe".

It is extremely hard to prove such code correct in Rust.

Re:The struggle with Rust by dbaupp on Wednesday January 25, 2017 @02:29PM (#13485069)
> I meant not having to manage different memory models and available access patterns

The only way I can make this make sense, is having cross-platform abstraction around atomic orderings etc. (like the new C++11 std::atomics) that works on x86 and ARM and PowerPC etc.

Rust has such a cross-platform abstraction, almost identical to C++11: https://doc.rust-lang.org/std/sync/atomic/index.html

Please be more specific about what exactly you mean if I am wrong.

Re:The struggle with Rust by braveo on Tuesday January 24, 2017 @02:46PM (#13474791)
> Do you really need to use malloc to allocate memory in a systems language? Do you really need to use pointer arithmetic to deal with things in a systems language? Sometimes, but not necessarily often. Even the Rust operating systems have managed to minimize the amount of unsafe/C-like code.

If you're adding performance considerations I think the answer is going to be yes.

Re:The struggle with Rust by bryanlarsen on Tuesday January 24, 2017 @03:13PM (#13475024)
I'm not an expert, but I believe that Rust's performance will be better if you avoid unsafe pointers. Rust's abstractions are generally cost-free, but if you use unsafe pointers the compiler will have to avoid optimizations that could be broken by potential pointer aliasing.
Re:The struggle with Rust by braveo on Tuesday January 24, 2017 @03:35PM (#13475222)
the idea of cost-free came from C++ where the belief is that you don't pay for what you don't use. The underlying meaning there is that you don't pay the cost of virtualized function calls if you're not using polymorphism (this is a big reason why C++ doesn't feature virtualized function calls by default).

polymorphism is of course an abstraction, the idea of cost-free is that you only pay for the abstractions you actually use.

This is not the same as maximizing performance.

The real question is whether or not you can do these things without giving the developer the control to maximize performance, and I think the answer is going to be no. The response will be that it's "good enough", but that's not actionable as we don't know what "good enough" means for any particular project.

Re:The struggle with Rust by Manishearth on Tuesday January 24, 2017 @04:03PM (#13475415)
> If you're adding performance considerations I think the answer is going to be yes.

You can build (or use, from the stdlib) safe abstractions over malloc that have the same performance. This should cover 99% of the use cases of malloc. For the few quirky cases where you still need malloc, you can still use it, but that means that the language doesn't need to make malloc-based memory management easy.

Re:The struggle with Rust by braveo on Tuesday January 24, 2017 @05:36PM (#13476420)
I find that unfair. C++ has new as well as other mechanisms such as perfect forwarding into smart objects, but no one seems to think C++ qualifies for the "you can do systems programming without malloc and pointers"?

You can't do systems programming without them, what you can do is minimize their use by building abstractions, which other languages, such as C++ and D, already do.

So this idea that people somehow think you have to use the actual malloc call throughout your code is a bit of a strawman.

Re:The struggle with Rust by Manishearth on Tuesday January 24, 2017 @07:12PM (#13477132)
I do think that C++ qualifies for that. I don't think there is widespread use of this yet -- I'm talking about the vast majority of "systems" code in C or C++ -- most of it is older and doesn't tend to use abstractions, or has been written from a "C with goodies" perspective.

So yeah, (modern) C++ also contests this claim too. Rust contests a stronger claim about safety as well.

------

You're the one who said that "it's claimed to be a systems level language, and if it's really that painful to do systems level things," -- my point is that in both Rust and C++, "systems level things" don't need to involve pointer arithmetic and other similar things. They should be niche tools you reach for when the abstractions fail you, and the abstractions should not fail you that often. This is already true for both languages, I feel. It's not "niche" in C++ because the ecosystem needs time to evolve to that point, but for new code it probably works out.

Re:The struggle with Rust by braveo on Tuesday January 24, 2017 @07:32PM (#13477256)
like I said, that's a strawman. C++ has had new/new[]/auto_ptr since C++ 98, so almost 20 years. The ideas of smart pointers have been around for a really long time as well.

Then there's the C constructor convention through opaque void pointers idiom.

This stuff is not new, not even close to new, NO ONE, not even the kernel developers, would make the argument that you cannot build abstractions around malloc.

If that's really what you're arguing, well you're not saying anything that C developers don't already practice.

Re:The struggle with Rust by Manishearth on Tuesday January 24, 2017 @07:54PM (#13477408)
When I say "malloc", I'm including `new` -- programming patterns that use new/delete are the same with malloc/free. new/delete doesn't really change the programming pattern, it just has a slightly nicer API.

smart pointers have been around for a while too. I see some codebases using them. But many don't.

> NO ONE, not even the kernel developers, would make the argument that you cannot build abstractions around malloc.

I never said that kernel devs or whatever would make that argument.

----------------

Look, you seem to think I've said something different than what I did here.

You said "if it's really that painful to do systems level things, then there's a problem somewhere.". I'm saying that Rust contests the claim that malloc/new/whatever must be easy to use for systems level things. Easy-to-use abstractions around them work well too. It's not that C++ doesn't have these abstractions, it does, and has for a while. My statement isn't one against C++.

My point is that the difficulty of using programming paradigms designed around malloc/new should not be a dealbreaker for a systems language, provided that the same tasks can be accomplished without raw allocation using a suitably generic abstraction. C certainly promotes malloc-based-paradigms. C++ has good alternatives, but a lot of the lower-level stuff I find avoids them because ultimately malloc/new aren't hard in C++. Rust makes them harder to use, but provides sufficient abstractions/abstraction-building-power that this should not be an impedance to systems programming. And, again, I'm not saying C++ doesn't -- I'm just saying that C++ also makes it easy to use malloc/new-based patterns, so you will continue to see those in systems programs even if they aren't necessary.

Now, interestingly, an abstraction for a vec-with-header doesn't actually seem to exist in the crate ecosystem, probably because folks don't like unconditional allocation. But it's relatively easy to build, might do it over the weekend.

(You can fake it in safe code using DSTs but it's not so great)

Re:The struggle with Rust by braveo on Tuesday January 24, 2017 @09:10PM (#13477816)
new/new[], delete/delete[] are abstractions around *alloc/free. That's not really under discussion, that's what they are. So when you talk about the ability to build abstractions around malloc, you're also talking about these operators in C++.

You may not like that they tend to be very thin abstractions, but they are abstractions, and the point is that the idea of building abstractions around malloc isn't new and certainly isn't unique.

You said:

>> You can build (or use, from the stdlib) safe abstractions over malloc that have the same performance.

While in defense of the statement:

>> Rust contests the claim that you actually need to do pointer arithmetic and stuff often in systems languages. You need to do it a bit to implement your abstractions, maybe, but it is not the first tool you should need to reach for.

This is presented as something unique to rust, which isn't even close to the accurate. This is why I called it a strawman. No one, not even kernel developers using C, are incapable of building abstractions around malloc to make it easier/safer to use.

The rest of what you say is inconsequential in my opinion. This isn't about C++, this is about your implication that building abstractions around malloc is something new or novel about Rust. It isn't. Not even the idea of safety around malloc is new, new[]/delete[] are themselves abstractions intended to make it less error prone to create/destroy contiguous blocks of memory.

That doesn't even get into the smart objects and RAII which have been around for a very very long time in C++.

None of this is new to Rust, and I think it's a bit disingenuous to imply that it is.

Re:The struggle with Rust by Manishearth on Tuesday January 24, 2017 @11:26PM (#13478511)
> So when you talk about the ability to build abstractions around malloc, you're also talking about these operators in C++.

Right, I wasn't talking about just that, I was specifically referring to abstractions that make you change your programming patterns. C++ has those too (the move-based abstractions), but I don't consider new/delete to be them.

My point was sort of "being able to write code that looks like this is not necessary for systems programming". Being able to do the task at hand is, but it need not be done using code like that.

> This is presented as something unique to rust, which isn't even close to the accurate.

Fair. It's not. I didn't intend to mean that. I'm sorry.

I was a C++ programmer for a quite a while, some of which time I used modern C++, so I'm quite aware that these abstractions are not unique to Rust.

The reason I phrased it that way is that C++ doesn't really make it harder to use direct-malloc-based (or raw-pointers-with-new) programming, and most of the lower-level C++ code I've seen (this is anecdotal) still uses this. So C++ doesn't contest the claim the same way Rust does (it does contest the claim though), since Rust codebases doing the same thing (even the kernels in Rust) tend to move strongly in the direction of avoiding unsafe pointer twiddling as much as it can. This is a bit more nuanced (and based on anecdotal evidence on low level codebases I've seen) than I really wrote in my original comment, and I apologize for that.