"RustConf 2017 - Closing Keynote: Safe Systems Software and the Future of Computing by Joe Duffy"
https://www.youtube.com/watch?v=CuD7SCqHB7k
"Safe Systems Programming in C# and .NET"
https://www.infoq.com/presentations/csharp-systems-programmi...
In one of them, he mentions that even with Midori proving its value to the Windows team, they were quite dimissive of it.
It appears to have also been yet another victim of the usual DevDiv vs WinDev politics.
I will say, unequivocally, that Spectre actually makes process isolation in single address space operating systems impossible on modern hardware. There is too much speculation and too many leaks, and it's not just branches. We wrote a whole paper about it a few years back.
For example all code from a particular vendor (origin) could share a single MPK whilst running, even if the failure domain for things like fault isolation is finer grained.
That isn't enough, because you can induce misspeculation through paths that do (or would) have access to appropriate MPKs and do almost anything you want, including disclosing information through sidechannels you do have access to. Loading an MPK is akin to changing address space protections; it has to be a hard barrier that cannot be speculated through. You cannot even have code mapped that would have access to those MPKs, as you can induce misspeculation into this code.
> It's an open research question but my gut says yes.
My gut says no. There is just too much dark stuff going on in hardware. Sidechannels are outside of models. You can't verify anything until you have not the model, but the whole design of the chip.
Also, variant 4 is not addressed much in the literature. I couldn't write what I wanted to write because of NDAs, but I have personally written PoCs that basically say hardware has to turn off memory speculation or you end up with a universal read gadget again. There is no software solution for variant 4.
I think there are also other caveats to consider. A lot of Spectre research (like your paper) is implicitly focused on the web renderer/V8 use case but here we're discussing theoretical operating system designs. Let's say you sandbox an image decoder library in-process, using type security and without using MPKs. Is this useless? No, because even if the image decoder is:
a. Maliciously doing speculation attacks.
b. Somehow this doesn't get noticed during development.
... the sandbox means it won't have any ability to make architectural-level changes. It can spy on you but its mouth is sealed; it doesn't have the ability to do any IO due to the architectural sandbox. To abuse Spectre in this context would require something really crazy, like trying to speculatively walk the heap, find the data you're looking for, encrypt it, steganographically encode the result into the images it decodes, and then hope that somehow those images make it back to the attackers even though the destination is probably just the screen. This isn't even NSA-level stuff, it's more like Hollywood at that point.
Compare to the situation today: the image decoder (or whatever) is a buggy C library running in a process with network access because it's a web origin. Game over.
I worry that the consequence of Spectre research has been that people conclude "in-process sandboxing is useless, cross-process is too hard, oh well, too bad so sad". Whereas in reality in-process sandboxing even without MPKs or equivalent would still be a massive win for computer security in many contexts where Spectre is hardly your biggest problem.
Well, I agree that in-process sandboxing is still quite useful; it at least closes the barn door. But the rest of the conclusion is not what we made in Chrome; we had to go whole-hog multi-process for site isolation. That and just moving as much as possible out of the renderer process so that there aren't many secrets left to steal.
It's really an issue for situations where a process (or platform) is required to run untrusted code from lots of different sources. There isn't a software solution that is robust to side channels yet. They can still spy on each other. Clearly, two import cases that Google cares about are Cloud and the web.
Both Smalltalk and LISP were used to write operating systems decades ago.
There were clear compromises & performance degradations related to using a GC'ed language, but it definitely was interesting enough that you hope for more.
Is there a situation where if your sticking points could. E addressed or tolerated where using a GC'd language could shine vs a non one?
I think there are many cases where GC'd languages can benefit tasks traditionally thought of as 'systems' tasks (I don't personally recognize much distinction between systems programming and non-systems programming). The Graal project is probably the most successful in this area. Like Midori it came out of long term corporate R&D, but unlike Midori it successfully shipped impactful software. They also published papers which the Midori guys never did.
Graal is an implementation of a JIT and AOT compiler for many languages, both source code and byte code based. It can run not only JVM bytecode but e.g. Python, Ruby, LLVM bitcode, WASM, and a whole bunch more, which makes a JVM powered by it easily the most polyglot compiler and runtime systems in history. Languages aren't translated to JVM bytecode, which would introduce impedance mismatches, but rather input to the compiler via a rather interesting technique called partial evaluation of interpreters.
Graal came out of asking the question, what if we wrote a JVM in Java? How can we do that, and if we can, are there benefits? Well, just rewriting something in Java doesn't give any benefits to the end users, just the developers, so to justify this you really have to find ways to really boost productivity a lot and then use that to yield interesting end-user features as well. Midori seems to have failed at this, Graal succeeded.
IMO the primary benefits Graal gets out of being written in a high level statically typed managed language are:
1. All the usual benefits like GC, great libraries, rock solid refactoring IDEs.
2. Annotations and the associated annotation processing/reflection infrastructure. The part that makes Graal polyglot is called Truffle and Truffle relies on this feature extensively.
3. Medium-level program input representation. One thing the Java world got really right is the clean separation between frontend and backend compilers via a stable bytecode format. Bytecode is not too high level, so you don't have to think about the complexities of evolving syntax when reading it, but also not too low level. Graal is of course itself expressed using bytecode and the compiler exploits this all over the place, for instance, it can parse bits of itself into compiler graphs at runtime and then use them as templates. They call this snippets and they use it as a way to lower high level constructs into lower level ones. It's really neat and a boost to compiler development productivity.
4. Related to that, Truffle relies heavily on that sort of meta-circular reflection capability. Truffle is the way Graal can compile languages that aren't JVM bytecode based. You write an AST interpreter for the language in any language that can produce bytecode (but in reality it's always Java because you need fairly precise control over the output; Kotlin might also work). The interpreter uses the Truffle API to express the AST, in particular it uses lots of annotations. The resulting interpreter is a normal Java program that can run on anything that can interpret bytecode, but the Graal compiler has plugins that recognize when it's compiling a Truffle program and handle it in special ways.
This ability to construct large, complex API surfaces that trigger special compiler behavior is one of the huge productivity wins that allowed the Graal team to add so many languages with such high runtime performance, so fast and so cheaply. It's like compiler intrinsics but taken to the next level, and then the next level again. And the end result is real benefits delivered to end users, for instance, Shopify uses TruffleRuby to get better performance for components of their web app.
You can do these things with WASM, which has no GC and simply manages memory segments on a per-module basis.
WASM cannot do this because it's meant to be a target for C and C-like languages. Either you have to make one WASM address space contain all your software, in which case it can easily corrupt itself internally (the sandbox is useless because it contains everything), or, you have to re-introduce a process-like concept that gives you isolated memory spaces, at which point it's not a single address space OS anymore.
Because then it can work? It's a matter of what virtual addresses each "realm" has access to, either reading, writing or both.
If there are two realms or protection domains or whatever we want to call them, but there is memory protection in place to prevent reading/writing of others when one is active, you can pass a pointer from one to the other and the other knows it's not belonging to itself. But the moment that receiver tries to read it, it'll segfault. Or what are you imagining happens here?
It seems to me like to solve that you have to copy data, not pointers. Now you have marshalling.
There's a second problem with trying to solve this with WASM - C and the associated ABIs don't handle evolution all that well. But part of what you need in a single address space OS is the ability for components to evolve independently, or semi-independently. In particular you need the ability to add fields to structures/objects without needing to recompile the world. Ideally, you'd even be able to modify structures in memory without even restarting the software. Higher level VMs than WASM can do this because they provide higher level semantics for memory layouts and linkage. You can evolve a JAR in much more flexible ways than you can evolve a binary C/rust module, or at least it's a lot less painful, which is why Win32 is full of reserved fields, and most attempts at long-term stable C APIs are full of OO-style APIs like GObject or COM in which structs are always opaque and everything has to be modified via slow inter-DLL calls to setters.
Perhaps it's worth stepping back. The reason SASOSs are always written in managed languages like C# or Java [dialects] is that they're trying to solve several problems simultaneously:
1. IPC is slow due to all the copying and context switching.
2. Beyond slow it's also awkward to share data structures across processes. You need SHM segments, special negotiations, custom allocators that let you control precisely where data goes and which are thread safe across processes etc. Even if you do it, you need a lot of protocols to get memory management right like IUnknown. So in practice it's rarely done outside of simple and special cases like shared pixel buffers.
3. Hardware processes conflate several different things together that we'd like to unpack, such as fault isolation, privacy, permissions etc.
4. Hard to evolve data structures when code is compiled using C ABIs.
and so on.
Simply creating non-overlapping address spaces doesn't help with any of these things. Even if all you do on a context switch is twiddle permission bits, it doesn't matter: you still need to do a TLB flush and that's the bulk of the cost of the context switch, and at any rate, you can't just call a function in another address space. Even if you know its address, and can allocate some memory to hold the arguments and pointers, you can't jump there because the target is mapped with zero permissions. And even if you go via a trampoline, so what, the stack holding your arguments also isn't readable. If you fix that with more hacks, now any structures the arguments point to aren't readable and so on recursively.
So you end up having to introduce RPC to copy the data structures across. Well, now what if you want a genuinely shared bit of state? You need some notion of handles, proxies, stubs, and that in turn means you need a way to coordinate lifetime management so different quasi-processes don't try to access memory another process freed. That's COM IUnknown::AddRef. Then you need ways to handle loosely coupled components that can be upgraded independently. That's COM IUnknown::QueryInterface and friends. And so on and so forth.
In a SASOS all that goes away because the compiler and GC are tightly integrated, and they don't let you manufacture arbitrary pointers. You don't have to do refcounting or marshalling as a consequence, you can evolve the ABIs of components without breaking things, you can create capabilities easily and cheaply, and so on.
As discussed above, speculation is a pain because it lets you break the rule of not crafting arbitrary pointers, but there are caveats to that caveat. I'm not actually convinced Spectre kills SASOS though you do need to do things differently.
As for what the benefit is, I think you can at the very least get rid of needing to copy data back and forth.
Not that I'm an advocate for single address space OS's. I'd have to think about this more. You might be right. I'm playing devil's advocate to think it through, not to defend a position, if that makes sense.
Well, pages are coarse grained so you'd end up giving the other quasi-process access to more stuff than it should have. And you'd have to flush the TLB so you pay the context switch cost anyway, at which point why bother? The reason operating systems make overlapping mappings is (classically) to enable better memory sharing due to not needing relocations or GOT/PLTs. That's all irrelevant these days due to ASLR (which doesn't even work that well anymore) but that was the idea.
You can do some tricks with using special allocators for the stuff you want to reveal that places the data in SHM segments, then blit the stack across to the other process and it can work. I know this because I built such a system as my undergrad thesis project :)
https://web.archive.org/web/20160331135050/plan99.net/~mike/...
It's very ugly though. And again, you need to coordinate memory management. For FastRPC it didn't matter because it was used mostly for holding stuff temporarily whilst you called into a library, so the 'outer' process owned the memory and the 'inner' process just used it temporarily. I never did anything with complex lifetimes.
One way of thinking about it is to study the issues that cause people to write large JVM apps instead of large C apps. It's not any different at the systems level. They want GC that works across all the libraries they use, they want to be able to upgrade the backend compiler without frontend-recompiling everything, they want to be able to upgrade a library or change a memory layout without recompiling the world, and they don't want all the goop that is triggered by allowing libraries to use arbitrary compilers and memory management subsystems. Forcing a unified compiler and GC simplifies the inter-module protocols so drastically, the wins are enormous.
Good point. Embarrassing oversight on my part. My whole mental model of how this would work has come crashing down. Now obvious to me that to have it work the way I envisioned, you would need a managed-code only environment that supervises all memory accesses.
>I know this because I built such a system as my undergrad thesis project
Very cool!
Yes this is a problem with all operating systems and it is much less likely with Java compared to C but it feels to me like you are entirely reliant on the JVM producing "verified" code. It's kinda like how the Rust gang has to do formal verification to prove that their borrow checker is actually water tight instead of just acting as if the programmer made sure it is water tight like in C land.
The point of managed code is that it compresses the space where (memory) safety problems can creep in to a small core in the runtime, which gets really well tested and reviewed. Even if there are concurrency bugs in your user-level code they can't corrupt memory in the manner C code can do - just yield objects in an invalid state that will hopefully be detected and cause exceptions very quickly. In turn you can then recover and try again or let the user know. CoModExceptions are a good example of that in action.
I noticed in another thread that a few people seem to think you can't implement an entire operating system in a GCd language like Java or C#, but that isn't true. You can do it like this:
1. Use an ahead of time compiler to compile the base runtime/kernel image to static machine code. In managed language operating systems like JNode or Singularity there isn't a strong distinction between kernel and language runtime, so this is the same thing. This base needs to contain at least a GC and probably a JITC, as well as some basic support services. This can itself be written in the managed language.
2. Write a very short bootstrap loader in assembly like every OS has, which sets things up enough to jump into the entry point of that runtime.
3. Writing a compiler in a managed language is easy enough but what about a GC? To do this you teach the compiler that some methods are 'magic' and shouldn't be treated as normal method calls. Instead they become either compiler intrinsics that are translated to pure assembly e.g. to read/write raw memory locations, or they are compiled in special ways for example to remove GC safe points.
The current best example of this in action is the GC in SubstrateVM, which is the "VM" compiled into any program AOT compiled with the GraalVM native image tool:
https://github.com/oracle/graal/tree/master/substratevm/src/...
If you flick through it you'll see various annotations and types you wouldn't normally see in Java, like `Pointer` and `@Uninterruptible`. These are recognized by the compiler and affects how the machine code is generated. The language is the same, so all existing tools continue to work - it's not a dialect or subset of Java, it's the full thing, just with slightly modified rules for how the final generated code behaves.
SubstrateVM has one more trick up its sleeve to break the circularity: some objects can be initialized and persisted to an "image heap" at build time. In other words, the GC code can use Java classes to structure itself, despite being unable to allocate.
And that's all it needs.
There have been efforts to do things like this in the past for full operating systems. They have nice properties: for example you can sandbox drivers, IPC overheads goes away because it's all in a single address space, capabilities actually work and are pervasive, and it's quite easy to keep ABIs stable. There are usually a few sticking points that prevent them taking off:
1. Historically, GCs have either been good at latency or throughput but not both simultaneously. Some of them also had trouble with large heaps. That's a problem because virtually all reasonable computing requires a mix of programs with wildly different sensitivity, most obviously, developers want latency to be prioritized for editing in their IDE but throughput to be prioritized for their build system. If you have one GCd heap for the entire computer then you have two options:
1a. Pick one algorithm to manage it.
1b. Do what Singularity did and come up with a quasi-process notion in which each unit has its own GCd heap. Singularity had a nice concept called an 'exchange heap' which allowed objects to be passed between these quasi-processes very fast and cheaply, whilst ensuring that object graph could only be pointed to by one unit at once. This made IPC tremendously cheap, allowed one unit to be paused for GC whilst other units ran, and let them use IPC all over the place. However it did reduce the benefits of using managed languages somewhat as it reintroduced complex data ownership rules.
NB: This is changing now with tech like HotSpot ZGC and Shenandoah (which are written in C++ though). They drive latencies through the floor, it's basically pauseless, and the new not yet released fully generational variants have very good throughput too. Also, G1 GC has monstrous throughput even with low pause times, they just aren't quite as low as ZGC/Shenandoah.
2. Overheads of managed languages are higher. The successor at MS Research to Singularity was codenamed Midori and not much was ever published about it publicly, but from what was written (by Joe Duffy) it seemed apparent that they went down a rabbithole of trying to make Midori have the same raw efficiency and overhead as C++ based Windows. They got a long way but ended up not having any interesting enough new features to justify the investment and the project was eventually canned.
3. All the same problems new operating systems always have: no apps, drivers etc.
4. Spectre attacks make single address space operating systems more complex. The new support for memory protection keys in Intel CPUs could re-awaken this area of OS research however because MPKs let you block speculation attacks within a single address space.