If anything is misleading it's linking to random emails in lkml without context. :-P (Maybe that's what you were getting at).
Personally, I'm very excited about Rust in the kernel.
Why?
- It would make me 10x more likely to work on the kernel, just because I enjoy programming in rust more than I enjoy programming in C. (At least given my current employment, any kernel contributions would be on my own time)
- It would give me more faith in the security of other code people are contributing, things like binder in C strike me as pretty scary components of android from a security perspective, I'm much more comfortable running the same thing written in Rust.
- I think it would generally reduce the number of kernel bugs in components written in it. Kernel bugs are rare, but and very frustrating when you encounter them.
- I think it would increase the general productivity in kernel development. A better kernel helps everyone out.
- It acts as validation for rust as a language. Obviously the kernel people shouldn't care about this in the slightest and it's not an argument for including it. However if it is included for other reasons (see above), it does help me argue that "rust would be a good fit for x" in other situations.
Seconding this. I don't think I would ever touch the Linux kernel with a twenty-foot-pole in C. In Rust? Maybe I'd give it a try and contribute one day.
Knowing the exact set of constraints on my code, having them enforced, not having to hold the entire system in my head just to avoid a catastrophic memory error. These things make contributing to a project - especially one as incomprehensibly large and complex as the Linux kernel - orders of magnitude easier.
a bit confused about this.
Rust talks the C ffi really well. It can call external functions that follow the C abi the same way it can call native unsafe functions [1]. You can tell it to layout a struct the same way that C does, etc.
Because calling the C abi requires unsafe code, it's common to provide wrappers around the C abi that are safe against missuse. I.e. that make it so that the only way to call C functions is the correct way. This is doing things like making it so the only way to get a `struct` is to call a (safe) `new` function that calls the (unsafe) C initializer internally, and exposing the C "methods" on that struct (that expect it to be initialized) as safe methods on the rust struct that internally call the unsafe C functions (and they can do so because they know the struct has been initialized). Obviously for any particular C api you have to look at what it requires to be called safely, and then figure out how to encode it in the type system, but that's usually surprisingly easy.
Calling rust from C doesn't really require any "unsafe" code (other than the fact that C is basically a giant unsafe block by nature), because the assertion that you're calling it correctly happens on the C side of things, not the rust side of things. Just like rust can call C abi functions, rust can make it's functions follow the C abi by simply saying
extern "C" fn foo()
fn foo()
[1] So what unsafe here means that the compiler doesn't know that the function is safe to call, so you have to tell it "I checked and how I'm using it is fine" by putting the call inside an unsafe block. This looks like the following. Note that you can also have unsafe native rust functions (e.g. if you want to index an array without checking the array bounds that's an unsafe function implemented in rust)
unsafe {
c_function_here(arg1, arg2)
}but let's say one is writing a filesystem in rust, so you're implementing most of the functions in "struct file_operations", and moreover you are passing "struct inode" , "struct page" etc ... back and forth between c and rust. with such heavy handed interaction, aren't we basically doing c in rust by necessity of the interface? by which i mean "unsafe" the way you defined it?
are there examples where you see a clear win?
My assumption would be that a file system is calling the same methods on a few different objects repeatedly. E.g. "read me some bytes from this page" or "get the id of this inode". For each of these APIs you once write a small amount of unsafe code that encodes into the type system "and this is how you can call it safely", and then you repeatedly get to make use of that code with guarantees that you aren't making any mistakes that are too terrible (logic bugs still exist obviously, which on a file system could delete or corrupt files, but you aren't going to corrupt some random kernel memory by accident). That's a pretty big win in my mind.
Meanwhile file systems probably include a lot of non-ffi things I think rust is substantially better for too. Like handling of a ton of different error's (oh no, the disk failed to give me bytes. Oh no, these bytes make no sense. etc) in the codes "happy"(ish) path. And like parsing data structures out of bytes (correctly). Tracking exclusive access to various resources. Implementing compression algorithms. Etc.
The case where you would see the sort of issue you're discussing is where all the code is doing basically unique ffi calls, so you don't get any reuse out of safe abstractions. I don't know of any great examples of this, maybe things like boot sequence code where you're running a lot of unique things exactly once to initialize the hardware?
now let me not impose on your kindness further and go learn a little rust.
Google is developing a bluetooth stack in Rust.
[1] https://blog.desdelinux.net/en/google-desarrolla-una-nueva-p...
Don't you think the massive increase in compilation time would negate any productivity gains and probably decrease productivity overall?
While slow compile times may slow you down, and hence reduce your productivity, if the compiler prevents hard to fix bugs, it still may increase your overall productivity. Consider things like https://hacks.mozilla.org/2021/04/eliminating-data-races-in-...
> Overall Rust appears to be fulfilling one of its original design goals: allowing us to write more concurrent code safely. Both WebRender and Stylo are very large and pervasively multi-threaded, but have had minimal threading issues. What issues we did find were mistakes in the implementations of low-level and explicitly unsafe multithreading abstractions — and those mistakes were simple to fix.
>
> This is in contrast to many of our C++ races, which often involved things being randomly accessed on different threads with unclear semantics, necessitating non-trivial refactorings of the code.
Maybe the C++ was faster to compile, but in the end, fixing these issues took more time. There were more of them, and they were harder to track down.
Nobody truly knows the answers to these questions in the general case yet, of course. My point is just that "slow compile == bad productivity" is not inherently true.
Faster compile times are, of course, always desired no matter what.
We can look at history and results. In these terms, C (and perhaps C++ to some extent) is, I believe, the only productive programming language for making the low level parts of non-experimental kernels.
As far as it can be publically measured, Rust so far has proven itself for application programming in a certain niche and not much more. It would be cool if it could prove itself in kernel space too -- we certainly need less system crashes caused by bad kernel-level code. Curiously though, it has been a very long time since I've last bumped into such a thing in Linux. This makes me suspect that Rust is trying to fix a problem here that's already been fixed in another way.
As for ease of coding, C seems like a massively easier language to learn than Rust. But I might be wrong there. Any data about that, I wonder?
> Any data about that, I wonder?
Possibly one of the only things harder to measure than productivity is ease of learning, haha! I had programmed in C for decades before Rust even existed. We do have a lot of people who say that they think Rust was easier to learn for them than C was. And of course many who believe the opposite. Not sure anything is conclusive in any direction. For example, it's quite possible that some people find C easier, and some people find Rust easier, and there will never be a clear winner.
Time will tell.
I don't know if this is what you mean or not, but I've seen a lot of claims that a language is "easier to learn" that seem to be considering "learning a language" as a valuable topic on its own, separate from "learning to write and maintain correct nontrivial programs in the language", and that seems wrong to me.
There's a part of this idea that does seem valuable to me, in that at the beginning of your learning process, there are a lot of benefits from being able to quickly get to a point where you can successfully write small programs that do something. It helps your motivation. It helps you reach some amount of productivity faster. When a language is better at early onboarding, it's more-useful to people who have smaller needs and more-constrained use-cases. Python being so easy to learn to glue together some libraries makes it a fantastic, valuable tool for many people.
The part of this idea that I really disagree with is how it applies to non-trivial, non-beginner use-cases. There are topics and skills that languages vary in their coverage of, but that you still need to learn about and deal with anyway for many types of programs. Memory management, resource handling, ownership and sharing, concurrency, nullability, error handling, composition, organization, abstraction, refactoring, testing, debugging, etc. A language including more or less that directly addresses these topics doesn't necessarily mean you won't still need to learn them.
To me, the relevant question isn't "Which language is easier to learn in isolation?", but instead "Which language is easier to learn to implement safe, performant, efficient, reliable, concurrent code with?".
If you take a new engineer who has "learned C", how easy is it to train them to get their rate of memory safety errors, thread safety errors, missed error checking, etc. down to the same rate as you'd get from a new engineer who has "learned Rust"?
Without tooling support like you get from Rust, you instead need to learn safe idioms, learn strategies to minimize your exposure to errors, train yourself to always always check everything at all times, learn how to write tests to discover mistakes you've made, learn how to use a collection of third-party tools you can use to approximate some of the benefits of Rust's compile-time checking, and train yourself to always use it. That's not "learning C", but it's still required in order to implement something like Linux.
Rust's bet is that there are ways to reduce the overall complexity of everything involved in implementing high-reliability high-performance systems by moving some of that complexity into the language. If you don't think it's accomplishing that goal, that's fine, but make that case directly.
I agree that the C programming language is smaller and easier to learn in isolation. It's not so obvious to me that something like "C + Valgrind + ASan + TSan + UBSan + ..." is easier to learn than Rust.
On the other hand, for many classes of errors that C offers no help with, Rust's compiler will directly point out where you've made a mistake, why it's wrong, and often offers advice on how to fix it. When learning a new language, having that kind of tooling support universally available is extremely helpful.
To be clear, Rust doesn't handle everything, and there's still a lot of benefit you can get from dynamic analysis tools, fuzzing, etc. There are also levels of reliability and assurance that aren't currently feasible with Rust. Rust has a long way to go.
The point I think I'm trying to make is that Rust really raises the bar in a meaningful way. There's some nonsense and awkward bits in Rust, but a lot of what you need to learn to be effective with Rust are things that you'd need to learn anyway to be effective at this level with C, and I think it's easier to learn those with Rust's help, and it's significantly easier to build systems with a much lower rate of these problems by using Rust.
Sorry for the length, and lack of organization. This has been rattling around in my head for a while, and I wanted to get some thoughts out in writing.
I have 10 years of writing Java on both small hobby projects and massive systems on my job. I dipped my toes into C/C++ a few times and I can write hobby project sized code that runs, but I can never really trust that code like I can the Java code that I write.
I had this issue especially when writing my first TeamSpeak Plugin: I could never be certain if I had to free the data or if TeamSpeak would do it for me after my function was called. I had to look into the documentation, as TeamSpeak is closed source.
About one and a half years ago, I finally sat down and learned Rust. I had tried to learn it several times before and always ran out of motivation.
This time, I got to hobby project level proficiency within a month or two. It was really fun and I rewrote several Java Spring projects in Rust's Rocket.
As Rocket starts just about instantly, even with the long compile times, the Rust rewrite takes less to compile than the Spring projects took to start up.
Coming to the point: I am already more confident about my Rust code than ever was about C and would even go so far as to say that I trust it more than my Java code, despite the big difference in experience.
I believe that I am ready to jump into a big Rust codebase and become productive as soon as I have understood the business-logic of the project in question.
I doubt I will be able to say the same about C/C++ in the near future.
That being said, using rust can be really nice for exploratory coding. If don't worry about edge case (use unwarp()/panic!) and don't worry about memory efficiency (use clone()) it still produces fast, memory efficient code.
- I don't think it will be massive.
- I especially don't think it will be that large for incremental builds, which is the main thing that matters. (But I'm not involved in this project, so I don't actually know how incremental the builds are...)
- My experience is that the vast majority of programming time is spent fixing mistakes, not compiling. Rust reduces the amount of time spent fixing mistakes a lot more than it increases time spent compiling.
- Rust moves many errors early in the compilation process (instead of when you try and test your code), which reduces iteration time instead of increasing it.
I'm not involved in this project, but I imagine it's at a point where you could get some numbers for the fixed overhead that adding rust adds to the compile times. I'd be interested in seeing those numbers.
2. A relatively small portion of code can account for the majority of exploited vulnerabilities.
3. Nothing about Rust is fundamentally slow with regards to compile times. There's plenty of time to work on it, and we've already seen significant efforts make headway.
Compile times are a nonissue in any practical sense for this work.
That sound was a thousand people nodding in unison
I have terrible news for you. People (well, humans, and I don't see anybody else signing up to maintain Linux) cannot in fact correctly handle pointers and goto. That's why they keep making mistakes.
It's actually to be hoped that Rust can usefully express constraints it has today to prevent some of those problems onto things like address spaces. It'd be great if say, driver code which can confuse a virtual address with a physical one just won't compile rather than compiling and then mysteriously not working as expected or causing occasionally BUG() reports.
Mitre shows that in fact people can't handle pointers and goto
But a memory allocation failure is clearly not a good reason.
> With the main point of Rust being safety, there is no way I will ever accept "panic dynamically" (whether due to out-of-memory or due to anything else - I also reacted to the "floating point use causes dynamic panics") as a feature in the Rust model.
- Kernels generally don't want to use floating-point, because saving and restoring the floating-point registers is fairly expensive. - Without some pretty aggressive hacking, it's not possible to remove floating-point support from Rust. - What you /can/ do (and what these patches do) is replace all the floating-point builtins with kernel panics. - This obviously sucks versus actually removing the floating-point, but this is an RFC.
For me, the main point is Can Rust be written to guarantee that no oom (or other hard fail 128-bit math) panic occurs that is not under control of the written code? I want the answer to be yes and also want to see how (which isn't far off from what Linus is asking).
Yes, easily, OOM is entirely a library created concept in rust, just don't use the standard library (or the `alloc` susbset of the standard library) and you don't have OOMs...
> (or other hard fail 128-bit math)
128 bit math, and floats, need to be avoided by just "not using them", the same as for floats in C code in the kernel...
Is it something about floating point operations not actually being fixed-sized?
Operating system kernels don't generally need floating point math (some of them use it anyway, lots don't).
So if your CPU has a way to say "Bottle your state, but er, don't worry about the floating point stuff" and it's faster, or uses less memory, or both, which are common, all the kernels (like Linux) which do not use floating point know they aren't touching that anyway and needn't bottle it up. This is potentially an important performance win.
If you try to take this performance win, but then you actually do use floating point in the kernel, the world suddenly changes beneath the feet of user tasks. A program is adding up some floating point numbers, and then, huh, suddenly the total is now negative? Wait, now it's zero? Nope, negative again? What's happening! If the program was aware of being interrupted this makes sense, but the whole point of pre-emptive multi-tasking is not to need to custom design every program to be interrupted everywhere.
So Linux (mostly) never uses floating point.
128 bit integers not working is a bit dumber, the compiler relies on some functions being implemented for them to work, and for whatever reason the people behind rust in linux just haven't implemented those functions (yet?). They use the normal (non floating point) registers though, so there is nothing stopping them from being implemented. https://github.com/Rust-for-Linux/linux/issues/11#issuecomme...
The main point of this RFC is that "the [in-kernel Rust] support is good enough that prototyping modules can start today." There's no point in making long arguments about alpha-quality design shortcuts on an alpha-quality prototype that are also explicitly mentioned by the patch authors to be of alpha quality.
See e.g. https://lkml.org/lkml/2021/4/14/1130 and https://lkml.org/lkml/2021/4/14/1023
EDIT: Thanks for the child comments; it seems that Linus is simply not aware of all the specifics and is asking for more information and/or decided to look at the code before reading the full mail thread.