News for hackers. Stuff that matters.
Re:Rust: Enable WebAssembly backend by default by cwzwarich on Sunday November 26, 2017 @01:24PM (#15782223)
> Consider https://os.phil-opp.com/page-tables/ for example, it uses the type system effectively to eliminate some issues, in the end, it only has two calls to unsafe.

However, those two calls to unsafe rely on an invariant maintained by the rest of the code that is not marked unsafe. Effectively, the entire module has to be treated as unsafe code by the programmer.

Re:Rust: Enable WebAssembly backend by default by steveklabnik on Sunday November 26, 2017 @01:27PM (#15782238)
Yes, absolutely. It's still a far cry from what many people assume, which is that the whole shebang must be.
Re:Rust: Enable WebAssembly backend by default by andrewflnr on Sunday November 26, 2017 @04:29PM (#15783101)
No, that's sort of the problem, isn't it? Safety actually does depend on the whole shebang, or rather, the correctness of all the code that supplies inputs to the "unsafe" block. I think safety has to be thought about at level of the unit of code that can enforce its own invariants, rather than the specific syntax that does the dirty work.
Re:Rust: Enable WebAssembly backend by default by steveklabnik on Sunday November 26, 2017 @04:38PM (#15783147)
Safety depends on any module containing unsafe code upholding the invariants that are required. But once you've done that, it's safe.

The question is, how much code is in this state, vs being purely safe? My experience and that of others shows that it's generally a smaller amount than many people not experienced with Rust assume.

Basically, I find the "there's always unsafe code somewhere at the bottom and therefore it's the same as 100% unsafe" to be overly reductive.

Re:Rust: Enable WebAssembly backend by default by frankmcsherry on Sunday November 26, 2017 @05:12PM (#15783349)
> Basically, I find the "there's always unsafe code somewhere at the bottom and therefore it's the same as 100% unsafe" to be overly reductive.

I totally agree; it's not the same. However, I think there is disagreement about whether saying only "there's always unsafe code somewhere at the bottom" is better or worse than saying "it's 100% unsafe".

I tend to write some sketchy unsafe code, and the only thing I can say with a straight face is "please don't use this if you care about safety", i.e. treat it as if "it's 100% unsafe". I'm slightly weirded out that Rust has a keyword that amounts to "I understand UB really well; tell others to trust me on it transitively".

I'll be less weirded out when the UB story gets shaken out more, or if cargo gets an "audit-unsafe" option that shows you all unsafe blocks you transitively depend on (does that exist yet?). Still like Rust, but letting randoms write unsafe code is scary.

Re:Rust: Enable WebAssembly backend by default by pcwalton on Sunday November 26, 2017 @08:41PM (#15784218)
> However, I think there is disagreement about whether saying only "there's always unsafe code somewhere at the bottom" is better or worse than saying "it's 100% unsafe".

Are you saying that Rust doesn't have a better memory safety story than C++? In practice, that hasn't been our experience.

Re:Rust: Enable WebAssembly backend by default by frankmcsherry on Sunday November 26, 2017 @09:50PM (#15784477)
I would say that Rust has a different memory safety story from say C#, which distinguishes between "is safe" and "contains unsafe code outside the CLR itself". I find that very-binary distinction more helpful than Rust's memory safety story.

I don't use C++ anymore, but I am not clear on how Rust has better memory safety guarantees than C++. I do like its story more, for sure, and I could see how they might turn in to guarantees (but I could also believe that someone could identify a similarly safe fragment of modern C++, and don't want to start that).

Edit: will rust-lang.org take a PR changing "guaranteed memory safety" to "memory safety story"? ;)

Re:Rust: Enable WebAssembly backend by default by burntsushi on Monday November 27, 2017 @08:14AM (#15787011)
> Edit: will rust-lang.org take a PR changing "guaranteed memory safety" to "memory safety story"? ;)

I hope not. The value proposition of Rust is that if you stick to writing code without unsafe, then you have a guarantee that you will not fall victim to memory unsafety unless there is a bug in the language or in some unsafe block in a dependency you're using.

Personally, I find your quibbling in this thread pretty strange. Rust very clearly provides more memory safety guarantees than C++. For starters, you could not take the guarantee I just stated above and apply it to C++ because there is no such thing as "C++ without unsafe blocks."

Re:Rust: Enable WebAssembly backend by default by frankmcsherry on Monday November 27, 2017 @09:22AM (#15787506)
I really don't have a position on C++; I don't use it, and don't really want to make supportive or limiting statements.

I have used other memory safe languages, and I find their guarantees qualitatively different from Rust. I would say of them "you will not fall victim to memory unsafety unless there is a bug in the language or in the standard library" with no mention of other dependencies. The standard library is then fairly thoroughly audited.

By comparison, you have e.g. the `memmap` crate, which I like and use, written by serious people known to you, where `unsafe` is used in construction of the memory map. It is then on the user to ensure that there are no concurrent writes to the backing file for the lifetime of the map, at the risk of (as I understand it) UB. Would you describe such a program as "safe"? Or at least, can you understand why I might not? If a concurrent modification happens, is the bug to be found in the `unsafe { MmapMut::map_mut(&file)? }` block, or perhaps farther away where you safely open and modify the file before dropping the map?

I'm sorry if this comes off as quibbling; I do actually care about getting these things right, as I want to rely on them and do more interesting things (e.g. de-abomonate mapped memory).

Re:Rust: Enable WebAssembly backend by default by pcwalton on Monday November 27, 2017 @12:04PM (#15788865)
C# has unsafe blocks just like Rust does. Likewise, Java has sun.misc.Unsafe. Python has ctypes. How is that different from Rust?
Re:Rust: Enable WebAssembly backend by default by frankmcsherry on Monday November 27, 2017 @01:01PM (#15789429)
HN wants me to stop posting, so this will be the last one. You two know how to find me elsewhere.

What I should have written is: "you will not fall victim to memory unsafety unless there is a bug in the language or in the standard library, or you explicitly opt out of safety". Using `unsafe` in C# is opting out of its safety guarantees (and requires the `/unsafe` flag); I'm pretty sure none of the C# team would say "guaranteed memory safety" of unsafe C#. I expect the same is true of Java, and have no clue about Python.

To the extent that we are talking about unsafe C#, I think you are right; it's mostly the same as Rust. But, no one would claim that unsafe C# has "guaranteed memory safety" where the guarantee is "as long as your code and all the code you bring in don't have memory safety bugs". At least, I hope not.

My original point was just meant to be that there is the risk of harm when you try and quantify a language as "more safe" because there are fewer regions where unsafe operations are permitted. People might read "guaranteed memory safety" and misunderstand what it means, as evidenced by me not actually knowing what it means.

Re:Rust: Enable WebAssembly backend by default by steveklabnik on Sunday November 26, 2017 @06:18PM (#15783657)
There is some tooling but not a ton. Check out the Miri link elsewhere in the thred; eventually I expect us to have amazing tools here. It’s gonna take some time though.
Re:Rust: Enable WebAssembly backend by default by andrewflnr on Sunday November 26, 2017 @07:19PM (#15783892)
Woops, I think I misunderstood the context and repeated what you were already saying. I have become that which I hate. :(
Re:Rust: Enable WebAssembly backend by default by cwzwarich on Sunday November 26, 2017 @01:34PM (#15782263)
For kernel-level programming, I think Rust is basically the same as C or C++ in this regard. I haven't seen a kernel written in Rust where I am convinced there is a submodule that can be treated as if its correctness doesn't impact the safety of the entire kernel.
Re:Rust: Enable WebAssembly backend by default by TheDong on Sunday November 26, 2017 @03:17PM (#15782759)
I don't understand this argument at all.

"Rust is not perfect, so it's basically the same as C"

The fact that "american fuzzy lop" and "syzkaller" have gone through the kernel and found dozens of use-after-free's (see https://github.com/google/syzkaller/blob/4bd70f/docs/linux/f...), shows that even incredibly carefully programmed C messes up in totally rote ways.

The basic cleanup code which is able to be exploited here would not be in an unsafe block if this were rust. In fact, because rust abstracts memory management out better, most of the cleanup code wouldn't exist at all.

Rust also allows better abstractions. For example, Rc<T> in rust is much easier to use without screwing up than the manually handled reference counters littered throughout the kernel. It seems obvious to me that numerous deadlocks in the kernel could be avoided by not hand-rolling refcounting everywhere.

You're right that there's still the potential with rust for some code to compromise the safety of the entire kernel.

But with rust that's a subset of code, not practically every single line of the whole thing.

It seems to me like memory corruption/use-after-free/etc only being able to happen in a small fraction of code is better than it being able to happen anywhere.

Re:Rust: Enable WebAssembly backend by default by zaarn on Monday November 27, 2017 @03:47AM (#15785792)
> Rc<T>

Rc requires memory allocation. When you boot a kernel you don't have that. Until you set it up, you can't use anything that requires a heap, only stack values.

The best solution I found is to use C code to allocate raw memory and then go through the elaborate process of setting up kernel address space, after which you setup a frame allocator, after which you can setup interrupts and finally use pagefaults to normally use kernel heap.

But wait there is more! During a pagefault interrupt you can't use the kernel heap in a writeful manner that is not inplace. Since a page fault during a page fault is a double fault and thusly your only option is to crash the kernel. You can only read from the kernel heap, modify in place in the heap and write to pagetables.

Same goes for some other interrupts. Which btw, violate a lot of ways Rust code likes to work. An interrupt is like a coroutine but you don't have a nice runtime that abstracts it for you, instead the IRQ just smashes into the active execution and let's hope you haven't held any locks to important kernel stuff at that moment because you will have to crack that lock open.

Re:Rust: Enable WebAssembly backend by default by TheDong on Monday November 27, 2017 @03:27PM (#15790905)
My point is really that rust allows you to build those abstractions, regardless of that specific one.

Sure, the kernel has reusable things like its embedded linked list etc, but they're not type-safe, and they can't represent ownership.

With rust's type-system, a generic rc-like thing could be created which is suitable for the world of the kernel, even if it's not the stdlib Rc.

Re:Rust: Enable WebAssembly backend by default by zaarn on Tuesday November 28, 2017 @02:42AM (#15795210)
You can't have RC until you have setup a heap. The stack is not a safe place to put these things.

Without any memory management, things get hard fast, especially considering you still need to parse the memory layout to find out which parts of memory are even usable.

Re:Rust: Enable WebAssembly backend by default by Dylan16807 on Wednesday November 29, 2017 @12:55PM (#15809085)
The original comment was "Rc<T> in rust is much easier to use without screwing up than the manually handled reference counters littered throughout the kernel."

That's heap vs. heap. It comes after the very brief stage you're mentioning.

Re:Rust: Enable WebAssembly backend by default by pcwalton on Sunday November 26, 2017 @08:39PM (#15784207)
Why is kernel programming any different from user-level programming here?

I tend to think that the driver and network level, which is where most problems in the real world tend to lie, is similar to userland in how often unsafe code gets used and the impact of unsafe code on the correctness of the kernel. In fact, the fact that you can write some drivers in userland on certain OS's--and they can crash without bringing down the whole OS--is strong evidence of this.

Re:Rust: Enable WebAssembly backend by default by zaarn on Monday November 27, 2017 @03:39AM (#15785756)
x86 is unsafe. Period. A lot of devices are unsafe. Period.

You can write some device drivers in userspace. But you can't handle page faults in userspace. Those need to be handled by the kernel and will be 100% unsafe because any mistake will lead to a certain and quick double or triple fault.

Kernel Level Programming is very very different to normal user-level programming. There is no malloc. There is no segmentation. Nothing prevents you from dereferencing a null pointer. In fact, in kernel-land a null pointer is valid memory. You don't have any of the forgiving properties of Ring3 assembly code in kernel-land.

Re:Rust: Enable WebAssembly backend by default by rcxdude on Monday November 27, 2017 @02:34PM (#15790415)
What you are describing is <10% of, say, the linux kernel. The rest of it reads very much like user-land code (and dereferencing a null pointer will often earn you a nice warning message if not a panic).
Re:Rust: Enable WebAssembly backend by default by zaarn on Tuesday November 28, 2017 @02:43AM (#15795213)
Those are usually the important 10% of a kernel that build the foundation of everything else.

Once you have VMM and Paging setup, preventing a null pointer deref is not hard (simply unpage 0x0)

Re:Rust: Enable WebAssembly backend by default by Rusky on Sunday November 26, 2017 @02:25PM (#15782505)
> a submodule that can be treated as if its correctness doesn't impact the safety of the entire kernel.

You don't have to go all that way to be better than C or C++. Rust can give you a submodule A whose correctness doesn't impact the safety of any uses of submodule B.

Re:Rust: Enable WebAssembly backend by default by cwzwarich on Sunday November 26, 2017 @02:59PM (#15782665)
If the correctness of submodule A impacts the safety of the entire kernel, then it impacts the safety of the uses of any other submodule, because you can't really contain the consequences of the unsafe behavior.
Re:Rust: Enable WebAssembly backend by default by Rusky on Sunday November 26, 2017 @04:25PM (#15783076)
> you can't really contain the consequences of the unsafe behavior.

That's true, but the property I described can still help you track down the root cause and limit the number of places that interact with it.