It is an implementation of the L4 microkernel API. Microkernel systems can be more reliable than monolithic kernels, but traditionally have the problem of high communication overhead between the services constituting the kernel. The Mach kernel (used by macOS and others) is an example of a known slow microkernel. L4 implementations have been demonstrating since the mid 90s that low overhead microkernels are possible.

I wonder why neither Minix 3 nor Fuchsia are using L4. (I guess they may have learned the main lessons from L4 and chose to write those few thousand lines of code themselves for flexibility and control. Strangely, nobody likes to talk about it.)

MINIX predates L4 and has evolved independently.

Unfortunately, L4 and microkernels in general have (unfairly) gained a bad reputation in some circles, stemming from some projects in the 90s that ended in spectacular failure. Examples include IBM's Workplace OS, which costed billions of dollars in development. L4 has had quiet success in the embedded space. General interest has renewed thanks to interesting projects like the seL4 project.

> I wonder why neither Minix 3 nor Fuchsia are using L4

There are many variants of L4, which have similar concepts and APIs but favour different design choices. So it's not that straight forward. In my opinion, I suspect companies with enough manpower will find it easier to make a new project and take the best ideas of L4 without importing all the controversial or philosophical aspects.

Edit: fixed a typo

"MINIX predates L4 and has evolved independently."

Minix 3, a new system with the same name, doesn't predate L4. Tannenbaum probably learned stuff from the L4 people given he cites their work in his microkernel debate with Torvalds. They made a custom kernel since their requirements were different than L4 people's. They might have also thought it would be fun. Lots of researchers like just doing their own thing for the experience.

"L4 has had quiet success in the embedded space. "

Don't understate it: OKL4 claimed deployment in a billion phones mostly for baseband isolation. Got acquired by General Dynamics for big bucks. So, there's one, success story.

Not big bucks, it was broke, general dynamics shut it down shortly after. Nobody with options got paid. The claim of a billion phones should be taken with some salt and l4 wasn't being used like a microkernel on the baseband - everything running in protected mode. If okl4 is the benchmark of success, microkernels are a near total failure. OKL4 was an abject failure, he said bitterly.

Oh wait, it looks like I missed his write-ups looking back at the company since I transitioned to a new area. I'll be going through them later. Thanks for the correction! So, I drop claims to L4 mostly a failure so far with other microkernels being successful in niche markets, esp embedded and safety/security-critical. The L4 people at places like Dresden at least gave us lots of good designs that are being put to use in open-source and commercial works. Device, driver VM's and Nitpicker GUI come to mind.

EDIT: Still reading the articles. Gernot reminds me PikeOS from Sysgo was "a L4 clone." They got acquired by Thales after quite a few iterations on their PikeOS product. An annual report I found by Thales said they were acquired for over 20 million Euros. Still on low-end in terms of market value but might have been successful depending on money in vs gained.

EDIT 2: Read his blog posts. Before OK Labs Post No 6, here's what he claims about L4 uptake:

"Secondly, a lot of stuff is running on L4: – Qualcomm modem chips have been running our L4-embedded kernel since 2006, that’s a few billion devices – The security processor of all iOS devices is running a modified version of our L4-embedded, that’s another 200-300 million devices every year – seL4 is being deployed on autonomous military air and ground vehicles, with the first autonomous helicopter flight in July 2015 – many other safety- and security-critical deployments are in progress.

I’m not aware of any real-world deployments of Minix."

Of course, we found out later Minix 3 was in Intel's baseband. So, there's two of them with people claiming massive deployments in supporting CPU's/MCU's. What you alluded to didn't refute a large number of units. Just that the company was anything as good as marketing presented. It wasn't. A few deliverables kept selling, though, per same source. He was wrong about Minix 3, too. Probably because it was Intel's secret for a while. I bet they wanted the reliability, included apps/code, and BSD license.

Definitely treat anything written by Heiser like any other marketing document, identify his interests at the time of writing first. And sure you can say that for almost anyone in this business...

Thanks for the overview. This is exactly the level of analysis I was looking for!

>The Mach kernel (used by macOS and others)

Where/how does this typically manifest itself? Why isn't macOS slow/sluggish?

While Mach 3.0 was designed to be a microkernel, macOS doesn't actually use it as one. The entire kernel, called XNU, runs in kernel mode, just like Linux.

The macOS kernel had a reputation for sluggishness back in its early days, due (to the best of my understanding) due to the sluggishness of Mach messaging, which is how Mach implements system calls to the kernel. macOS also supports BSD syscalls, and Apple has apparently done enough optimization here that it's now roughly on par with Linux.

Around 2006, Apple did have an internal, experimental version of macOS ("Darbat") which ran Mach on top of the L4 microkernel, but this project was canceled, for whatever reason.

Darbat seems to be what's running in the Secure Enclave of the iPhone. Or at least something like it.

https://www.blackhat.com/docs/us-16/materials/us-16-Mandt-De...

Pretty darned awesome.

Fiasco is GREAT. You at least used to be able to run it with DROPS which had a cool desktop/demo disk where you could just launch a bunch of Debian Linux instances as L4 tasks. It was like VMWare on crack at the time when I tried it.

I think FiascoUX might still exist... you can run Fiasco as a userspace process on linux then run linux in ... oh I've gone cross-eyed..

DROPS is still available here:

http://demo.tudos.org/eng_about.html

It was one, cool-ass demo. I especially liked firing up VM's lightening-fast given I was reading people say they use containers since VM's are too slow and inefficient. Maybe the people building their favorite VM's just aren't good at efficiency. Build on L4/L4Linux instead. ;)

The Secure Enclave runs a customized version of L4Ka::Pistachio, apparently. Different codebase (Pistachio is C++, for example), same origin.

The kernel of macOS is XNU, which is basically a hybrid of Mach (microkernel) and BSD (monolithic Kernel). I guess the implication is that they had to create this hybrid because it would have been too slow otherwise.

I suspect you'll find the reason for a large amount of BSD code in macOS is because it allows them to get a lot of stuff for free that they didn't otherwise have to write themselves.

>Why isn't macOS slow/sluggish?

That's a feature of the reality distortion field.

Andrew Tannenbaum goes into detail in his exploration on how to make operating systems reliable:

https://www.openu.ac.il/home/wiseman/2os/microkernels/tanenb...

In a debate with Torvalds, he also cites a lot of examples of microkernel use:

https://www.cs.vu.nl/~ast/reliable-os/

Although most are commercial, Genode is an example of an open-source system built with the architecture. It's a variation of Nizza architecture whose idea was to minimize attack surface and complexity to just what a specific application needs.

https://os.inf.tu-dresden.de/papers_ps/nizza.pdf

https://genode.org/

Somebody feel free to set me straight if I'm wrong (my knowledge of kernels is basically a college class and a half), but it seems the key thing here is that it's a kernel oriented around the L4 cache, which seems interesting because that's shared by the CPU and GPU: https://superuser.com/questions/1073937/what-does-l4-cache-h...

So, intuitively at least it seems it's for sharing the kernel workload with your GPU (supposing your hardware supports it)?

Ehm, no, it has nothing to do with the cache, it's just a successor of L3[0].

[0] https://en.wikipedia.org/wiki/L4_microkernel_family#L3

Ah, thanks. Hadn't heard of it, but now I know!