""" Normally, when you write a brand new kernel driver as complicated as this one, trying to go from simple demo apps to a full desktop with multiple apps using the GPU concurrently ends up triggering all sorts of race conditions, memory leaks, use-after-free issues, and all kinds of badness.
But all that just… didn’t happen! I only had to fix a few logic bugs and one issue in the core of the memory management code, and then everything else just worked stably! Rust is truly magical! Its safety features mean that the design of the driver is guaranteed to be thread-safe and memory-safe as long as there are no issues in the few unsafe sections. It really guides you towards not just safe but good design. """
https://asahilinux.org/2022/11/tales-of-the-m1-gpu/
> the whole thing seems kinda cute but like, shouldn't this experiment in programming language co-development be taking place somewhere other than the source tree for the world's most important piece of software?
Torvalds seems to disagree with you.
i don't really care for mindless appeals to authority. make your own arguments and defend them or don't bother.
this gpu driver looks pretty cool though. looks like there's much more to the rust compatibility layer in the asahi tree and it is pretty cool that they were able to ship so quickly. i'd be curious how kernel rust compares to user space rust with respect to bloat. (user rust is pretty bad in that regard, imo)
You previously appealed to Linux being the largest worlds most important source tree and then you choose to eschew the opinion of that project's lead.
Unless you have a specific falsifiable claim that is being challenged or defended, it's not at all a fallacy to assume expert opinions are implicitly correct. It's just wisdom and good sense, even if it's not useful to the debate you want to have.
This isn't an appeal to just any authority, but the authority that defines Linux and is its namesake.
Surely it's better to cite the authority's reasons as to why they think this way than just to cite the authority itself ...
It's not like there's a lack of times he's talked about rust. Just link his commentary similar to linking ashai linux.
Why? When disagreeing with an authority, you want the audience to pay closer attention to your arguments as you demonstrate why the authority has it wrong. When you're just sharing distant and likely under-informed opinions with no arguments to back them up, it's not up to other people to do homework to show you why you're wrong. Appeal to authority is a legit call to a fallacy only when people give next to no consideration to your arguments, focusing instead on the opposing party's stature.
Speaking seriously, they surely meant data races, right? If so, what's preventing me from using C++ atomics to achieve the same thing?
Rust’s strict ownership model enforces more correct handling of data that is shared or sent across threads.
> Speaking seriously, they surely meant data races, right? If so, what's preventing me from using C++ atomics to achieve the same thing?
C++ is not used in the Linux kernel.
You can write safe code in C++ or C if everything is attended to carefully and no mistakes are made by you or future maintainers who modify code. The benefit of Rust is that the compiler enforces it at a language level so you don’t have to rely on everyone touching the code avoiding mistakes or the disallowed behavior.
Rust inherently models this idea. Read about Rust's "Send" and "Sync" marker traits. e.g. https://doc.rust-lang.org/std/marker/trait.Send.html
> Is rust going to synchronize shared memory access for me?
Much better than that. (safe) Rust is going to complain that you can't write the unsynchronized nonsense you were probably going to write, shortcutting the step where in production everything gets corrupted and you spend six months trying to reproduce and debug your mistake...
i can't remember the last time i faced a data race to be honest.
i guess the real question is, how well does it all hold up when you have teamwork and everything isn't strictly adherent to one specific philosophy.
Spatial memory safety is easy, just check the bounds before indexing an array. Temporal memory safety is easy, just free memory only after you've finished using it, and not too early or too late. As you say, thread safety is easy.
Except we have loads of empirical evidence--from widespread failures of software--that it's not easy in practice. Especially in large codebases, remembering the remote conditions you need to uphold to maintain memory safety and thread safety can be difficult. I've written loads of code that created issues like "oops, I forgot to account for the possibility that someone might use this notification to immediately tell me to shut down."
What these annotations provide is a way to have the compiler bop you in the head when you accidentally screw something up, in the same way the compiler bops you in the head if you fucked up a type or the name of something. And my experience is that many people do go through a phase with the borrow checker where they complain about it being incorrect, only to later discover that it was correct, and the pattern they thought was safe wasn't.
Rust has real improvements here, like this example from the fuschia team of enforcing lock ordering at compile time [0]. This is technically possible in C++ as well (see Alon Wolf's metaprogramming), but it's truly dark magic to do so.
https://a10nw01f.github.io/post/advanced_compile_time_valida...
Given how the committee works and the direction they insist on taking, C++ will never ever become a safe language.
Bit of a fun fact, but as one of the linked articles states the C++ committee doesn't seem to be a fan of stateful metaprogramming so its status is somewhat unclear. From Core Working Group issue 2118:
> Defining a friend function in a template, then referencing that function later provides a means of capturing and retrieving metaprogramming state. This technique is arcane and should be made ill-formed.
> Notes from the May, 2015 meeting:
> CWG agreed that such techniques should be ill-formed, although the mechanism for prohibiting them is as yet undetermined.
"Just" annotations... that are automatically added (in the vast majority of cases) and enforced by the compiler.
> proper use of mutexes and lock ordering aren't that hard, they just require a little bit of discipline and consistency.
Yes, like how avoiding type confusion/OOB/use-after-free/etc. "just require[s] a little bit of discipline and consistency"?
The point of offloading these kinds of things onto the compiler/language is precisely so that you have something watching your back if/when your discipline and consistency slips, especially when dealing with larger/more complex systems/teams. Most of us are only human, after all.
> how well does it all hold up when you have teamwork and everything isn't strictly adherent to one specific philosophy.
Again, part of the point is that Send/Sync are virtually always handled by the compiler, so teamwork and philosophy generally aren't in the picture in the first place. Consider it an extension of your "regular" strong static type system checks (e.g., can't pass object of type A to a function that expects an unrelated object of type B) to cross-thread concerns.
No, they are not. You also don't need mutex ordering as much since Mutexes in Rust are a container type. You can only get ahold of the inside value as a reference when calling the lock method.
Mutex as a container has no bearing on lock ordering problems (deadlock).
Wow, you must be really smart! I guess you don’t need rust. For the rest of us who find concurrent programming difficult, it is useful.
Nothing is preventing you from writing correct C++ code. Rust is strictly less powerful (in terms of possible programs) than C++. The problem with C++ is that the easiest way to do anything is often the wrong way to do it. You might not even realize you are sharing a variable across threads and that it needs to be atomic.
Well, pretty close to that, actually! Rust will statically prevent you from accessing the same data from different threads concurrently without using a lock or atomic.
> what's preventing me from using C++ atomics to achieve the same thing
You might forget?
Imagine this C++ code:
class Foo {
// ...
public:
void frobFoo();
};
Now, is it okay to call `frobFoo` from multiple threads at once? Maybe, maybe not -- if it's not documented (or if you don't trust the documentation), you will have to read the entire implementation to answer that.Now imagine this Rust code:
struct Foo {
// ...
}
impl Foo {
fn frobFoo(&mut self) {
// ...
}
}
Now, is `frobFoo` okay to call from multiple threads at once? No, and the language will automatically make it impossible to do so.If we had `&self` instead of `&mut self`, then it might be okay, you can discover whether it's okay by pure local reasoning (looking at the traits implemented by Foo, not the implementation), and if it's not then the language will again automatically prevent you from doing so (and also prevent the function from doing anything that would make it unsafe).
Yes
> I.e. it assumes that all other code is Rust for the purpose of those checks?
Not exactly, it merely assumes that you upheld the documented invariants when you wrote code to call/be-called-from other languages. For example that if I have a `extern "C" fn foo(x: &mut i32)` that
- x points to a properly aligned properly allocated i32 (not to null, not to the middle of un-unallocated page somewhere)
- The only way that memory will be accessed for the duration of the call to `foo` is via `x`. Which is to say that other parts of the system won't be writing to `x` or making assumptions about what value is stored in its memory until the function call returns (rust is, in principle, permitted to store some temporary value in `x`s memory even if the code never touches x beyond being passed it. So long as when `foo` returns the memory contains what it is supposed to). Note that this implies that a pointer to the same memory isn't also being passed to rust some other way (e.g. through a static which doesn't have a locked lock around it)
- foo will be called via the standard "C" calling convention (on x86_64 linux this for instance means that the stack pointer must be 2-byte aligned. Which is the type of constraint that is very easy to violate from assembly and next to impossible to violate from C code).
That it's up to the programmer to verify the invariants is why FFI code is considered "unsafe" in rust - programmer error can result in unsoundness. But if you, the programmer, are confident you have upheld the invariants you still get the guarantees about the broader system.
Rust is generally all about local reasoning. It doesn't actually care very much what the rest of the system is, so long as it called us following the agreed upon contract. It just has a much more explicit definition of what that contract is then C.
This is fairly narrow, often C functions for example aren't actually safe, for example they take a pointer and it must be valid, that's not inherently safe, or they have requirements about the relative values of parameters or the state of the wider system which can't be checked by the Rust, again unsafe. But there are cases where this affordance is a nice improvement.
I like the term "checked" and "unchecked" better but not enough to actually lobby to change them, and as a term of art they're fine.
As far as I can tell, ANY guarantee provided by ANY language is "just a language construct" that fails if we assume there is other code executing which is ill-behaved.
Rust does, successfully, guarantee the lack of data races. It also guarantees the lack of memory-unsafety resulting from race conditions in general (which to be fair largely just means "it guarantees a lack of data races", though it does also include things like "race conditions won't result in a use after free or an out of bounds memory access").
If by address it you mean "show how C/C++ does this"... they don't and this is well known.
If by address it you mean "prove that rust doesn't do what it says it does"... as that point you're inviting someone to teach you the details of how rust works down to the nitty gritty in an HN comment. You'd be much better off finding and reading the relevant materials on the internet than someones off hand attempt at recreating them on HN.
Sadly, I don't know rustlang, so I can't tell if the inability to describe its features in more commonly used terms is due to incompetence or the features being irrelevant to this discussion (see the title of the thread).
As near as I can tell to give you the answer you're looking for I'd have to explain the majority of rust to you. How traits work, and auto traits, and unsafe trait impls, and ownership, and the borrow checker, and for it to make sense as a practical thing interior mutability, and then I could point you at the standard library concepts of Send and Sync which someone mentioned above and they would actually make sense, and then I could give some examples of how everything comes together to enable memory safe, efficient, and ergonomic, threading primitives.
But this would no longer be a discussion about a rust language feature, but a tutorial on rust in general. Because to properly understand how the primitives that allow rust to build safe abstractions work, you need to understand most of rust.
Send and Sync (mentioned up thread) while being useful search terms, are some of the last things in a reasonable rust curriculum, not the first. I could quickly explain them to someone who already knew rust, and hadn't used them (or threads) at all, because they're simple once you have the foundation of "how the rest of rust works". Skipping the foundation doesn't make sense.
† "Memory safety" was admittedly possibly popularized by rust, but is equivalent to "the absence of undefined behaviour" which should be understandable to any C programmer.
Well, yes, but that's the whole value of Rust: you don't need to use these overly-cautious defensive constructs, (at least not to prevent data races), because the language prevents them for you automatically.
The point is rather that it’s not. The “trait send sync things” specify whether a value of the type is allowed to be respectively move or borrowed across thread boundaries.
It won't prevent all races, but it might help avoid mistakes in a few of em. And concurrency is such a pain; any such machine-checked guarantees are probably nice to have to those dealing with em - caveat being that I'm not such a person.
I can only tell you: open your mind. Is Rust just a fad? The latest cool new shiny, espoused only by amateurs who don’t have a real job? Or is it something radically different? Go dig into Rust. Compile it down to assembly and see what it generates. Get frustrated by the borrow checker rules until you have the epiphany. Write some unsafe code and learn what “unsafe” really means. Form your own opinion.
This is incorrect. The android binder rust rewrite is planned to wholly replace the current C implementation.
https://www.phoronix.com/news/Rust-Binder-For-Linux-6.18
And most of the big drivers written for Apple M-series hardware are written in Rust, those are not simple rewrites or proof of concepts.
Vestigal how? The commit message in the version in Linus's tree (from commit eafedbc7c050c44744fbdf80bdf3315e860b7513 "rust_binder: add Rust Binder driver") makes it seem rather more complete:
> Rust binder passes all tests that validate the correctness of Binder in the Android Open Source Project. We can boot a device, and run a variety of apps and functionality without issues. We have performed this both on the Cuttlefish Android emulator device, and on a Pixel 6 Pro.
> As for feature parity, Rust binder currently implements all features that C binder supports, with the exception of some debugging facilities. The missing debugging facilities will be added before we submit the Rust implementation upstream.
----
> shouldn't this experiment in programming language co-development be taking place somewhere other than the source tree for the world's most important piece of software?
Rust for Linux did start as an out-of-tree project. The thing is that no matter how much work you do out of tree if you're serious about trying integration out you'll have to pull in experimental support at some point - which is more or less what is happening now.
It'll go faster once all the bindings are in place and people have more experience with this stuff. I've been greatly looking forward to expanding bcachefs's use of rust, right now it's just in userspace but I've got some initial bindings for bcachefs's core btree API.
Real iterators, closures, better data types, all that stuff is going to be so nice when it can replace pages and pages of macro madness.
The avalanche has already started. It is too late for the pebbles to vote.
0: https://blog.regehr.org/archives/1180 1: https://blog.regehr.org/archives/1287
Even if it is though, we don't have it. It seems like linux should go with the solution we have in hand and can see works, not a solution that hasn't been developed or proved possible and practical.
Nor is memory safety the only thing rust brings to the table, it's also brings a more expressive type system that prevents other mistakes (just not as categorically) and lets you program faster. Supposing we got this memory safe C that somehow avoided this complexity... I don't think I'd even want to use it over the more expressive memory safe language that also brings other benefits.
† A memory-safe managed C is possible of course (see https://fil-c.org/), but it seems unsuitable for a kernel.
†† There are some other alternatives to the choices rust made, but not meaningfully less complex. Separately you could ditch the complexity of async I guess, but you can also just use rust as if async didn't exist, it's a purely value added feature. There's likely one or two other similar examples though they don't immediately come to mind.
Can you give an example? One that remained a low level language, and remained ergonomic enough for practical use?
> Second, I do not even believe that memory safety is that important that this trumps other considerations
In your previous comment you stated "a memory safe C would be far more useful. It is sad that not more resources are invested into this". It seems to me that after suggesting that people should stop working on what they are working on and work on memory safe C instead you ought to be prepared to defend the concept of a memory safe C. Not to simply back away from memory safety being a useful concept in the first place.
I'm not particularly interested in debating the merits of memory safety with you, I entered this discussion upon the assumption that you had conceded them.
They can't, of course, because there was no such language. Some people for whatever reason struggle to acknowledge that (1) Rust was not just the synthesis of existing ideas (the borrow checker was novel, and aspects of its thread safety story like Send and Sync were also AFAIK not found in the literature), and (2) to the extent that it was the synthesis of existing ideas, a number of these were locked away in languages that were not even close to being ready for industry adoption. There was no other Rust alternative (that genuinely aimed to replace C++ for all use cases, not just supplement it) just on the horizon or something around the time of Rust 1.0's release. Pretty much all the oxygen in the room for developing such a language has gone to Rust for well over a decade now, and that's why it's in the Linux kernel and [insert your pet language here] is not.
BTW, this is also why people being are incentivized to figure out ways to solve complex cases like Rcu-projection through extensible mechanisms (like the generic field projection proposal) rather than ditching Rust as a language because it can't currently handle these ergonomically. The lack of alternatives to Rust is a big driving factor for people to find these abstractions. Conversely, having the weight of the Linux kernel behind these feature requests (instead of e.g. some random hobbyist) makes it far more likely for them to actually get into the language.
> But it seems kind of random why they picked Rust. I do not think there is anything which makes it particularly good and it certainly has issues.
Like I said, they picked Rust because there was literally no other suitable language. You're avoiding actually naming one because you know this is true. Even among academic languages very few targeted being able to replace C++ everywhere directly as the language was deemed unsuitable for verification due to its complexity. People were much more focused on the idea of providing end to end verified proofs that C code matched its specification, but that is not a viable approach for a language intended to be used by regular industry programmers. Plenty of other research languages wanted to compete with C++ in specific domains where the problem fit a shape that made the safety problem more tractable, but they were not true general purpose languages and it was not clear how to extend them to become such (or whether the language designers even wanted to). Other languages might have thought they were targeting the C++ domain but made far too many performance sacrifices to be suitable candidates, or gave up on safety where the problem get hard (how many "full memory safety" solutions completely give up on data races for example? More than a few).
As a "C++ guy" Rust was the very first language that gave us what we actually wanted out of a language (zero performance compromises) while adding something meaningful that we couldn't do without it (full memory safety). Even where it fell short on performance or safety, the difference with other languages was that nobody said "well, you shouldn't care about that anyway because it's not that big a deal on modern CPUs" or "well, that's a stupid thing for a user to do, who cares about making that case safe?" The language designers genuinely wanted to see how far we cold push things without compromises (and still do). The work to allow even complex Linux kernel concurrent patterns (like RCU or sequence locking) to be exposed through safe APIs, without explicitly hardcoding the safety proofs for the difficult parts into the language, is just an extension of the attitude that's been there since the beginning.
And, it's really funny that GP criticizes Rust but doesn't acknowledge that of course blood, sweat, and tears have already gone into less drastic variants for C or C++. Rust itself is one of the outputs of the solution space! Sure, hype is always a thing, but Rust has quite demonstrated its utility in the free market of programming languages. If Rust was not as promising as it is, I don't see why all of these companies and Linus Torvalds would seriously consider it after all these years of experience. I can accept if C had a valid "worse is better" merit to it. I think C++, if anything, has the worst value-to-hype ratio of any programming language. But Rust has never been a one-trick pony for memory safety, or a bag of old tricks. Like any good language, it offers its own way of doing things, and for many people, its way is a net improvement.
I agree that memory safety is useful, but I think the bigger problem is complexity, and Rust goes in the wrong direction. I also think that any investment into safety features - even if not achieving perfect safety - in C tooling would have much higher return of investment and bigger impact on the open-source ecosystem.
> This post is a long-winded way of saying that I lost faith in my ability to push the work forward.
The gem of despair:
> Another example is what should be done when a 32-bit integer is shifted by 32 places (this is undefined behavior in C and C++). Stephen Canon pointed out on twitter that there are many programs typically compiled for ARM that would fail if this produced something besides 0, and there are also many programs typically compiled for x86 that would fail when this evaluates to something other than the original value.
Since this is a Git repo, I'd go with `git ls-files '.rs'`.
That’s an interesting idea. You should let this guy know you disagree with his technical decision -> [email protected]
The language co-development isn't unique to Rust. There are plenty of features in GCC and Clang that exist specifically for Kernel usage too.
The fact that Rust gets to benefit from the project too is just an added bonus.
Because people want to use Rust where they use C, right now? Whereas yours is a perfectly fine criticism, it ignores that people want the good stuff, everywhere, in the things they actually use every day. And since this is something the project lead wants to do, this doesn't seem to problem/live issue.
Even figuring out what exactly the Linux kernel API safety rules are is a huge task, much less encoding them in a computer-readable form.
The code there is not about C<->Rust FFI. It's about encoding Linux kernel API properties into a safe Rust API.
The uncertainty of the calling/ordering rules is exactly why kernel C has been hard to write. For VFS locking rules, you pretty much have to simulate Al Viro's brain and replay his whole life experience...
i just decided do a good ol' 'find -name "*.rs"' in the kernel tree to get a sense for what all this is about. from what i can tell, there's just an api compatibility layer (found in /rust) and then a smattering of proof of concept drivers in tree that appear to just be simple rewrites of existing drivers (with the exception of the incomplete nvidia thing) that aren't even really in use. from what i can tell even the android binder rust rewrite is vestigial.
the whole thing seems kinda cute but like, shouldn't this experiment in programming language co-development be taking place somewhere other than the source tree for the world's most important piece of software?
redox is a pretty cool experimental piece of software that might be the os of the future, why not do it there?