News for hackers. Stuff that matters.
Re:Writing an OS in Rust: Handling Exceptions by bluejekyll on Sunday March 26, 2017 @12:39PM (#13961604)
I wish I had more spare time to help work on something like this. I've been reading all of these posts since the beginning, and they are all so excellent. I knew a lot of these things abstractly, but this explains it all in such solid detail, it's awesome. Thank you!

Now a question for people in the know about micro-kernel architectures. One thing often claimed is that the message passing is too expensive, causing essentially double the work per system call.

After reading this, I started wondering, and now want to read more about it: is it possible to use page faults and a handler to effectively build a message passing setup in the microkernel? Essentially using the pagefault as the mechanism to switch stacks from one kernel-service to another? Is this novel, stupid, or even faster than existing mechanisms?

Obviously I need to go read how L4 and others do this. What's great about posts like this is that it makes me want to go learn about it...

Re:Writing an OS in Rust: Handling Exceptions by the_duke on Sunday March 26, 2017 @01:39PM (#13961934)
The main performance problem isn't message passing itself but the required context switches.

With a monolithic kernel, with disk io as an example and presented in a very primitive form:

Userspace -> Kernel -> Userspace. Makes 2.

With a microkernel:

Userspace -> Kernel -> IO Service -> Disk Driver -> (IO Service) -> Kernel -> Userspace

Makes 5-6 context switches (at least) for a single disk read, since everything runs in an isolated process, and a context switch is quite expensive (saving & restoring registers, etc).

You can mitigate and optimize this though...

Re:Writing an OS in Rust: Handling Exceptions by Animats on Sunday March 26, 2017 @04:08PM (#13962930)
Moving data from one process to another by messing with the MMU state to move a page from one process to another has been tried. Mach did that. Unless you're moving megabytes of data, on modern CPUs, messing with the MMU state is usually a lose. Caches and translation lookaside buffers have to be flushed, which is worse than copying. Copying is a cheap operation on most modern CPUs, because the data paths are wide and the data you're sending is probably in cache. Also, the data has to be page-aligned for this to work.
Re:Writing an OS in Rust: Handling Exceptions by adamnemecek on Sunday March 26, 2017 @01:37PM (#13961914)
> Obviously I need to go read how L4 and others do this.

The L4 guy wrote a paper on it. https://www.cs.nyu.edu/~mwalfish/classes/15fa/ref/liedtke93i...

I think I've been intrigued by the same idea, I believe that Rust enables you to build a reactive OS which would simplify a lot of programming. Given that interrupts are basically events, it's a match made in heaven.

If someone wants to talk about this (by "this" I guess I mean the new OS architectures that are now possible due to Rust) more, shoot me an email (it's in my profile).

Re:Writing an OS in Rust: Handling Exceptions by mastax on Sunday March 26, 2017 @02:36PM (#13962300)
You may be interested in these projects:

https://github.com/ryanra/RustOS

RustOS is hobby OS project that's pretty bare bones, but has similar ideas to what you've mentioned. It's essentially a port of the Rust standard library to bare metal, relying on the safety of the language rather than hardware features. It's sort-of a unikernel in implementation if not intention.

https://github.com/helena-project/tock

Tock is an interesting academic project to create an embedded OS. It still uses more traditional hardware protection for user processes, since these can be written in any language. The kernel, however, is composed of cooperatively-scheduled components called capsules, which rely on the language for isolation and don't have the overhead associated with real processes. See this doc for more details: https://www.tockos.org/documentation/design/

Re:Writing an OS in Rust: Handling Exceptions by adamnemecek on Sunday March 26, 2017 @03:26PM (#13962612)
Both of those look really dope. How did you learn about them?
Re:Writing an OS in Rust: Handling Exceptions by kibwen on Sunday March 26, 2017 @03:54PM (#13962834)
reddit.com/r/rust is a useful resource for keeping abreast of Rust-related happenings.

For other OS stuff in Rust, see Redox (a microkernel) and Robigalia (Rust on seL4).

Re:Writing an OS in Rust: Handling Exceptions by adamnemecek on Sunday March 26, 2017 @05:30PM (#13963415)
I think that the thing I'm most interested in is what new architectures does Rust allow for.

Redox is cool but Unix has outlived it's usefulness at this point. Also. the Unix philosophy is the very anti-thesis of what I think is the Rust philosophy. I can imagine that a Rust port of QNX could be a big deal tho (QNX is/was only like 500KLOC of C, which could be like 100KLOC of Rust).

But I think that we need mode advanced systems, e.g. don't need as much virtual memory protection because that's handled by the language. There's actually quite a few things in the current OSs that seem to be pure anachronisms.

Re:Writing an OS in Rust: Handling Exceptions by bluejekyll on Sunday March 26, 2017 @06:31PM (#13963730)
> don't need as much virtual memory protection because that's handled by the language.

This is true at compile time. But what does it do for you at runtime when bad actors will be trying to break any and all assumptions? The guarantees in the kernel need to be vigilant against attackers, so while the kernel itself might be simpler, the boundary with processes still needs a lot of constraints at runtime.

Re:Writing an OS in Rust: Handling Exceptions by krzat on Monday March 27, 2017 @05:37AM (#13966230)
I think the assumption here is that OS would only allow binaries built from safe rust code.
Re:Writing an OS in Rust: Handling Exceptions by bluejekyll on Monday March 27, 2017 @11:18AM (#13968399)
Sure, but I don't know how you'd still guard against runtime issues. I'm not even sure you could require user space only safe code.
Re:Writing an OS in Rust: Handling Exceptions by MrBuddyCasino on Sunday March 26, 2017 @05:12PM (#13963324)
A Rust unikernel sounds very tempting for embedded stuff. Anyone doing that?
Re:Writing an OS in Rust: Handling Exceptions by nickpsecurity on Sunday March 26, 2017 @06:45PM (#13963802)
I'll add that it probably should be done. Aside from usefulness, it's already been done for Ada and embedded Java on top of partitioning microkernels. There's companies actually paying for these. The dynamic or concurrency safety of Rust might be appealing for some of them. Any money made there can go back into improving the project/product.
Re:Writing an OS in Rust: Handling Exceptions by qznc on Sunday March 26, 2017 @04:12PM (#13962958)
There is also this nice L4 overview paper "From L3 to seL4 What Have We Learnt in 20 Years of L4 Microkernels?"

http://sigops.org/sosp/sosp13/papers/p133-elphinstone.pdf

Re:Writing an OS in Rust: Handling Exceptions by milesrout on Sunday March 26, 2017 @10:01PM (#13964583)
I'm curious why you seem to think that Rust makes new OS architectures possible. Rust is a language, and languages have nothing to do with OSes. Do you mean new kernel structures?
Re:Writing an OS in Rust: Handling Exceptions by adamnemecek on Sunday March 26, 2017 @10:58PM (#13964801)
Because you can write more complex systems with it. E.g. asynchrony is much easier, therefore you can use it more frequently. Like you can write pretty complex kernel code without worrying about anything.

Similarly, some of the errors that are commonly handled by the OS are made impossible to express in Rust. Your OS can me made much leaner without loosing anything.

Re:Writing an OS in Rust: Handling Exceptions by milesrout on Monday March 27, 2017 @04:24PM (#13971411)
That's just not true. You don't get any of Rust's threading guarantees at the kernel level because a gigantic portion of your code is `unsafe` anyway, and has to be. (See Redox).

And 'some of the errors that are commonly handled by the OS' might be theoretically impossible to express in Rust, but they can still happen, so they still need to be handled.

Re:Writing an OS in Rust: Handling Exceptions by adamnemecek on Monday March 27, 2017 @05:18PM (#13971858)
> You don't get any of Rust's threading guarantees at the kernel level because a gigantic portion of your code is `unsafe` anyway, and has to be. (See Redox).

How do you not get "any"? You might lose some, because of unsafe code, but not all.

> And 'some of the errors that are commonly handled by the OS' might be theoretically impossible to express in Rust, but they can still happen, so they still need to be handled.

Sure but it's a fundamentally different situation. The types of errors might be pretty different.

Re:Writing an OS in Rust: Handling Exceptions by jacquesm on Sunday March 26, 2017 @03:56PM (#13962848)
> is it possible to use page faults and a handler to effectively build a message passing setup in the microkernel?

Yes. (Been there, done that, 20 years ago).

> Essentially using the pagefault as the mechanism to switch stacks from one kernel-service to another?

You wouldn't use the pagefault for that but you'd switch the context (TSS or equivalent), this you need to do anyway when you switch from one process to another, or from a process to the kernel. But passing the message could be done via paging.

> Is this novel,

No.

> stupid

No. Definitely not.

> or even faster than existing mechanisms?

It's not faster than the current syscall arrangement, but it is faster than copying data from one process to another, which is the way a message passing kernel would normally pass messages from one process to another.