News for hackers. Stuff that matters.
Re:Towards Linux Kernel Memory Safety by StudentStuff on Wednesday November 15, 2017 @11:32PM (#15710279)
So Rust it is then, eh? Could work through the kernel module by module to rewrite it in rust, but it'd be a multi-year process (like Firefox's rewrite has been).
Re:Towards Linux Kernel Memory Safety by ynezz on Thursday November 16, 2017 @01:58AM (#15710710)
You mean rewriting unsafe C code to Rust by annotating it with `unsafe` keyword? :-) Try to write in Rust something which would touch bare metal hardware. It would be PITA, it would be full of unsafe keywords and assembler code so it would possibly result in similar amount of bugs in the end, just in different parts.
Re:Towards Linux Kernel Memory Safety by dikaiosune on Thursday November 16, 2017 @02:38AM (#15710823)
This is empirically not true if you look at the usage of unsafe in pure Rust kernels. Yes, there is assembly needed, and some unsafe Rust code, but it's very possible to wrap up unsafe operations in safe APIs.

For example, the kernel repository for Redox (a written-in-Rust OS project) appears to have 242 usages of the unsafe keyword (including comments bc lazy), out of 18205 lines of Rust source.

Re:Towards Linux Kernel Memory Safety by zaarn on Thursday November 16, 2017 @04:16AM (#15711266)
It's actually not that much unsafe.

You can wrap most unsafe operations to be safe with proper checks, I've done it myself and it's much less than you would expect since a lot of the memory accesses can be made safe trivially.

Re:Towards Linux Kernel Memory Safety by red75prime on Thursday November 16, 2017 @03:19AM (#15710983)
I can't come up with a situation where C doesn't need inline assembly, while Rust does. Do you have some specific example on your mind?
Re:Towards Linux Kernel Memory Safety by Santosh83 on Thursday November 16, 2017 @06:56AM (#15711989)
Just to turn this discussion towards the practical aspect: does anyone know if a concrete effort has been made to sound out Torvalds and the other senior kernel devs about accepting Rust into the kernel?

I would imagine that initially as PoC one can start by implementing a simple module... obviously a small portion of the Rust stdlib and runtime needs to be ported and of course rustc needs to be integrated into the kernel toolchain...

Perhaps someone has done it privately (i.e., without attempting to submit patches back to source)?

Re:Towards Linux Kernel Memory Safety by steeleduncan on Thursday November 16, 2017 @07:17AM (#15712086)
In [1] (from 2016) he says

    I don't think you actually solve any of the really hard kernel problems with your choice of programming language.
    The big problems tend to be about hardware support
    (all those drivers, all the odd details about different platforms,
    all the subtleties in memory management and resource accounting)
I think what he is saying is that rewriting c->(e.g.) rust doesn't improve make progress on any of the design problems in the linux kernel.

In [2] (2013) he expands on that, essentially explaining how well C maps to machine instructions, and how that helps optimise the kernel.

Personally I think Rust maps very well to machine code, and although it doesn't solve design problems, I do think it saves a lot of time debugging the security flaws and bugs that can result from memory bugs.

[1] https://www.infoworld.com/article/3109150/linux/linux-at-25-... [2] https://www.reddit.com/r/programming/comments/1t0gfy/linus_t...

Re:Towards Linux Kernel Memory Safety by Santosh83 on Thursday November 16, 2017 @07:27AM (#15712128)
I tend to agree with his point that you may perhaps not solve design and driver problems just changing language, but can anyone dispute that a memory-safe language can at least help eliminate routine programming pitfalls like off-by-one, null deferences, double-free and so on? And these are the types of innocuous and inadvertent errors that are exploited by black-hats, sometimes with costly consequences.

I don't see a strong argument against a memory-safe language, although granted it won't solve architectural and hardware issues. Perhaps the effort to support, say Rust, in the kernel is just not worth the trade-off at this point? Perhaps no one has taken on the mantle for doing so? But I can't see how one can categorically argue against it...

Re:Towards Linux Kernel Memory Safety by agumonkey on Thursday November 16, 2017 @12:55PM (#15714803)
maybe a different approach to hardware programming, some kind of safer and higher abstraction system/language to improve quality and reduce effort ?
Re:Towards Linux Kernel Memory Safety by slrz on Thursday November 16, 2017 @07:59PM (#15718156)
> Just to turn this discussion towards the practical aspect: does anyone know if a concrete effort has been made to sound out Torvalds and the other senior kernel devs about accepting Rust into the kernel?

This is not how things work. You don't arrive at changes like this by out-talking Linus. Talk is cheap and Linus is opinionated.

You need to demonstrate that what you want to do is worthwile, i.e. is not only feasible but also provides a payoff that carries the weight of what you're proposing. You also need to be fiercely committed to your idea and to making it happen inside mainline Linux.

In other words, show me the code (and maintain it out-of-tree for some time, tracking upstream changes and gathering support from other developers).

Re:Towards Linux Kernel Memory Safety by empath75 on Thursday November 16, 2017 @07:25AM (#15712119)
Don’t think it’ll ever happen. More likely something like redox will get to the point that it’s production-ready and people will just stop using Linux.

I don’t think it’s far off, at least for vms, tbh.

Re:Towards Linux Kernel Memory Safety by sixbrx on Thursday November 16, 2017 @09:26AM (#15712789)
> at least for vms

Yeah that may be a really good upside to VMs, that they enable new OSs to have a more level playing field with the established OSs. Letting the VM layer handle all the messy details of real hardware.

Re:Towards Linux Kernel Memory Safety by IcePic on Thursday November 16, 2017 @04:14AM (#15711255)
..and of course Rust needs to be ported and have its .. "runtime" fixed for all platforms linux runs on. Otherwise this will be a decent way to kill off a bunch of platforms.
Re:Towards Linux Kernel Memory Safety by koffiezet on Thursday November 16, 2017 @10:17AM (#15713264)
Was going to point this out too... Having the Rust compiler support all platforms GCC is able to target is going to be a massive task - nevermind maintaining them.

That's one of the main advantages of C - it runs on pretty much everything.

Re:Towards Linux Kernel Memory Safety by ksec on Thursday November 16, 2017 @05:17AM (#15711529)
Well at least someone is already trying to write an OS with Rust.

https://github.com/redox-os/redox

Re:Towards Linux Kernel Memory Safety by pjmlp on Thursday November 16, 2017 @02:39AM (#15710829)
Not necessarily, Google's approach has been to forbid kernel access (ChromeOS), to make a big pain to use native code only for specific use cases (Android) or to develop a new microkernel in a mix of C++/Rust/Go/Dart (Fuchsia).
Re:Towards Linux Kernel Memory Safety by Rusky on Thursday November 16, 2017 @01:17PM (#15715048)
Fuchsia doesn't use Rust.
Re:Towards Linux Kernel Memory Safety by pjmlp on Thursday November 16, 2017 @03:03PM (#15716062)
Re:Towards Linux Kernel Memory Safety by Rusky on Saturday November 18, 2017 @02:19AM (#15728128)
Those are bindings.
Re:Towards Linux Kernel Memory Safety by Varriount on Thursday November 16, 2017 @04:28AM (#15711319)
Seriously? What is it worth people thinking that Rust is the only choice for such things? Memory safety is nothing new. Languages like D, Nim, and Swift all allow for the possibility of both memory safety and low-level access.
Re:Towards Linux Kernel Memory Safety by mkesper on Thursday November 16, 2017 @05:16AM (#15711525)
Memory safety without GC and with zero-cost abstractions is the combination rust enables. Nim and swift (and Go) strictly require a GC and D without GC doesn't work without heavy lifting.
Re:Towards Linux Kernel Memory Safety by dom96 on Thursday November 16, 2017 @05:23AM (#15711547)
> Nim and swift (and Go) strictly require a GC and D without GC doesn't work without heavy lifting.

I think Nim belongs with D here. You can freely disable the GC in Nim.

Full disclosure: I'm a Nim core dev.

Re:Towards Linux Kernel Memory Safety by zurn on Thursday November 16, 2017 @08:04AM (#15712313)
Refcounting and costly abstractions are in wide use currently - just done in C without the safety.
Re:Towards Linux Kernel Memory Safety by Varriount on Thursday November 16, 2017 @05:48AM (#15711665)
What's wrong with garbage collectors? Depending on the type, you can get performance characteristics better than or similar to destructor-based management.

Even current operating systems use forms of garbage collection (such as refcounting) to track resources.

Re:Towards Linux Kernel Memory Safety by cyphar on Thursday November 16, 2017 @08:02AM (#15712302)
The problems with a GC (in the context of embedding it in a larger codebase written in a another language) is that the calling conventions are more complicated and the existence of a GC (especially mark-and-sweep GCs) means that the embedding is not "seamless". Even with a C FFI interface it's still harder than (for example) using Rust or C in such a project.

And if both languages have GC, then you're in even more trouble.

Re:Towards Linux Kernel Memory Safety by johncolanduoni on Thursday November 16, 2017 @10:49AM (#15713606)
For what language runtime are the calling conventions an issue? Even the most calling-convention dependent GC I can think of (GHC Haskell's) can handle standard C ABIs and .NET is a good example of making this whole process relatively seamless. If you can export a C FFI with pinned pointers in both directions, I don't see how it's any harder than Rust or C.

The multiple GCs in a process issue is usually due to the collector not playing nice with signal/exception handler installation when using safepoints, which is purely an issue of lazy implementations. There's even ways to collect cross-heap cycles when working this multiple GCs (Xamarin on Android does this fairly well).

Re:Towards Linux Kernel Memory Safety by jhasse on Thursday November 16, 2017 @08:02AM (#15712299)
Most things get moved to the heap.
Re:Towards Linux Kernel Memory Safety by pjmlp on Thursday November 16, 2017 @08:46AM (#15712542)
Only in GC languages that don't offer good language features for value types, which usually isn't the case with the ones targeted for low level systems development.
Re:Towards Linux Kernel Memory Safety by Rusky on Thursday November 16, 2017 @01:20PM (#15715078)
"Good language features for value types," while retaining memory safety, means at least one of these things:

* Boxing anything you take a reference directly to

* Forbidding sum types and complicating the GC to allow interior pointers

* Forbidding references to value types

* A borrow checker

Guess which one is most flexible, and novel in Rust.

Re:Towards Linux Kernel Memory Safety by pjmlp on Thursday November 16, 2017 @01:39PM (#15715266)
A language like Modula-3 doesn't suffer from those issues.

We only don't have it thanks to Compaq killing the project after acquiring DEC Olivetti. Politics as usual.

Rust ideas come from Cyclone and ATS, it is not novel in Rust.

The borrow checker still is pretty much WIP, that the NLL changes will surely improve, but don't not yet solve all ergonomic problems.

Re:Towards Linux Kernel Memory Safety by Rusky on Saturday November 18, 2017 @02:20AM (#15728129)
Modula-3 either uses one of those techniques, or is not memory safe. (Looks like primarily the first one- manual deallocation is unsafe and it relies on GC for the rest?)

Cyclone and ATS do not have Rust's combination of ownership and borrowing.

Re:Towards Linux Kernel Memory Safety by pjmlp on Thursday November 16, 2017 @06:15AM (#15711804)
Ada already enabled it.
Re:Towards Linux Kernel Memory Safety by johncolanduoni on Thursday November 16, 2017 @10:31AM (#15713406)
Ada's model for safe allocations is far more limited than Rust's, however. Particularly in ways that would require more unsafe code when writing a kernel.
Re:Towards Linux Kernel Memory Safety by pjmlp on Thursday November 16, 2017 @10:51AM (#15713626)
I don't see why, given that Unchecked_Deallocation calls should only exist inside data structure implementations, never called directly from safe code.

Which should just use controlled types, pools or standard containers.

Also there are a few open experiments with Ada,

https://muen.codelabs.ch/

https://github.com/Lucretia/bare_bones

https://marte.unican.es/

Then there are the commercial ones used in the industry, making use of bare metal profiles like Ravenscar.

Re:Towards Linux Kernel Memory Safety by DigitalJack on Wednesday November 15, 2017 @11:54PM (#15710342)
it's 20 million lines of code.
Re:Towards Linux Kernel Memory Safety by staticassertion on Thursday November 16, 2017 @01:25AM (#15710612)
Right, but you don't have to rewrite it at once. There are certain areas of the kernel that have historically had more issues, are at higher risk of attack, etc. Those will make up far less code than 20 million lines, and you can start there.
Re:Towards Linux Kernel Memory Safety by monocasa on Wednesday November 15, 2017 @11:57PM (#15710349)
You could triage. Zorro bus support can probably wait until someone has a specific itch to scratch.
Re:Towards Linux Kernel Memory Safety by vilhelm_s on Thursday November 16, 2017 @12:01AM (#15710355)
Firefox is 17 million!
Re:Towards Linux Kernel Memory Safety by akmittal on Thursday November 16, 2017 @04:49AM (#15711420)
They rewrote 160,000 lines and it took them more than a year

> It replaces approximately 160,000 lines of C++ with 85,000 lines of Rust.

https://blog.rust-lang.org/2017/11/14/Fearless-Concurrency-I...

Re:Towards Linux Kernel Memory Safety by mundanevoice on Thursday November 16, 2017 @01:27AM (#15710618)
The whole Firefox codebase has not been ported to Rust and will not be anytime soon. So your argument doesn't hold true.
Re:Towards Linux Kernel Memory Safety by daveFNbuck on Thursday November 16, 2017 @01:58AM (#15710711)
Isn't that a point in the rewrite's favor? You don't have to port everything to gain a lot of the benefits.
Re:Towards Linux Kernel Memory Safety by mundanevoice on Thursday November 16, 2017 @02:02AM (#15710727)
> You don't have to port everything to gain a lot of the benefits.

I agree with this. But just saying Firefox has X millions of lines of code doesn't say anything about the possibility for Linux to be ported to Rust.

Re:Towards Linux Kernel Memory Safety by daveFNbuck on Thursday November 16, 2017 @02:46AM (#15710852)
It does say that the 20 million lines statistic isn't sufficient to dismiss the idea. That's not saying much, but I think it's a valid response to someone pointing out that the Linux kernel is 20 million lines.
Re:Towards Linux Kernel Memory Safety by craftyguy on Thursday November 16, 2017 @02:20AM (#15710770)
you're comparing apples and rutabagas. code line count is a hilariously inaccurate way to measure the complexity required for a portable OS kernel vs a userspace application.
Re:Towards Linux Kernel Memory Safety by mundanevoice on Thursday November 16, 2017 @03:26AM (#15711027)
> code line count is a hilariously inaccurate way to measure the complexity required for a portable OS kernel vs a userspace application

I never said that. My whole premise is that you can't say anything like this by just saying code count on two very different projects. Maybe you didn't read my comments properly.

Re:Towards Linux Kernel Memory Safety by mfukar on Thursday November 16, 2017 @05:50AM (#15711680)
What benefits, precisely, though? New bugs? http://insert-script.blogspot.de/2017/11/firefox-settings-co...
Re:Towards Linux Kernel Memory Safety by irishsultan on Thursday November 16, 2017 @06:28AM (#15711852)
What's the relation with Rust? I don't see it mentioned in the post, it affects Firefox before version 57, but not 57 (not that there wasn't any Rust code in Firefox before 57) and it seems to be a bug in DOMParser, which as far as I know is not written in Rust.
Re:Towards Linux Kernel Memory Safety by jhasse on Thursday November 16, 2017 @08:00AM (#15712283)
Better security. Oh wait ... Chrome still wins although it's written in C++.
Re:Towards Linux Kernel Memory Safety by rurban on Saturday November 18, 2017 @03:23AM (#15728235)
Rust is not safe. ATS, Pony, C#, Java, lisp come to my mind. But maybe even D, Nim, swift, crystal. Singularity/Midori or the LispM were good candidates. And it cannot be a monolithic kernel of course.