I imagine there's two pools of Linux users: those that compile their own kernel, probably with only the drivers they use; and those that use distribution kernels, which have a small set of compiled in drivers, and a huge initrd with all the drivers. Neither one of those groups ends up with all of the drivers loaded into memory: the compile their own have a small set, the distro kernel people end up with only the drivers they need loaded; both groups have a relatively small kernel footprint.
Have you looked at the size of L3 caches lately? You could fit a whole ‘nother OS in one of those.
Process isolation and selinux are things that exist. You even mention it. Turns out, having complete isolation between basically everything in your system is hard. I don't see how microkernels will make that difficulty disappear?
You can ask for numbers without casting aspersions like that.
This is HN. It is something like a casual conversation. You are free to ask for numbers, other people are free to provide or not provide them, and we should be able to have this conversation without leveling accusations of dishonesty (which the word “feels” does not really temper).
I was actually trying to find the numbers myself, and it was surprisingly more involved than I'd care for. I didn't want to give someone else more work, but happy to see the numbers expanded. I stand by my criticism, though. It feels off. If you want to say it is justified, spend more effort justifying it.
The phrase “it feels” is not carte blanche to say what you want.
I could say, “Someone reading this post might think that you’re an axe murderer.” Certainly not me, I don’t think you’re an axe murderer. But someone reading this post might think so. What this does is introduce a new topic of conversation—that topic is you, and whether you’re an axe murderer. I’ve done it without saying anything untrue. It is literally true, that someone reading this post might think that you’re an axe murderer. That has much the same effect as an accusation, but I’ve given you no real way to respond.
When you write “it feels”, there is an implied subject, who is feeling those feelings. That subject is you.
I flat out feel it is dishonest, in the vein that it is cherry picked data to further support a point that they agree with. Such that, using "this is a casual conversation" as a rubrik, I absolutely call BS from people when they say stuff like this to me.
If this was a "town square debate" where the conversation is partisan and folks are trying to score points, I'd agree with your assertion a bit more. But for casual conversation specifically, I ultimately reject this view.
Now, if the poster is specifically offended, I'm more than willing to apologize on that and to clarify my point. I had no ill things to say about the poster or the main topic. Heavily critical of cherry picked data, with a justification of why.
No, actually quite the opposite: Reductio ad absurdum is a form of logical proof, not a fallacy.
Process isolation actually isn’t fundamentally hard…it’s just hard on a system that wasn’t designed for it. And SELinux doesn’t do what microkernels do. Even with a tightly locked down SELinux configuration, a driver vulnerability can lead to root access. SELinux can protect your userland the same way that microkernels do, but it can’t put your drivers in userland.
I do question if process isolation isn't fundamentally hard. Separate processes for separate things entirely is not fundamentally hard. That, I agree. But, so much of what we do is integrated together. Is why early attempts to make it so that some programs can't see my whole hard drive are bonkers annoying. I specifically want access to the picture I just saved in that editing program so that I can email it.
In this case they’re both old though, as Hurd predates Linux by a year.
With drivers:
---------------------------------------------------------------------------------------
Language files blank comment code
---------------------------------------------------------------------------------------
C 32699 3332747 2610014 17218820
C/C++ Header 23690 712617 1373965 7109290
Assembly 1336 48534 101884 234314
---------------------------------------------------------------------------------------
Language files blank comment code
---------------------------------------------------------------------------------------
C 13200 1031558 951267 5356551
C/C++ Header 13760 300309 535100 1643611
Assembly 1321 47500 100899 230638
Numbers obtained by running cloc 1.96 on current kernel master.
I think many are misunderstanding my criticism here. I offer it more as a way to strengthen the message than I do to question it entirely.
To that end, the only "goal post shift" I would add is to ask how many of the 17 million lines of driver code have to be done for Hurd to support hardware that would make it more relevant for a ton of users?
Probably? If you really want an apples-to-apple comparison, delete everything but x86_64 support from both Linux and Hurd. Remove sample code.
linux/net isn't strictly "drivers", but implements the base of the networking stack. That's 900k lines. Is that networking base code included with the Hurd figures? If not, delete it (or include in your comparison Hurd's base networking code).
Beyond that, this is getting a bit ridiculous. Who cares about the LoC comparisons? They're very different kernels with very different levels of maturity and number of features. I'm not even convinced you can call one "better" or "worse" based on this metric at all.
Is that important, though? As soon as you start processing any meaningful amount of data, won't your OS be evicted from the cache anyway?
> unless you are compiling your kernel specifically for your machine, your kernel is going to be a bloated compromise of the set of drivers that your system is most likely to see. That means that you will have dozens, if not hundreds of drivers compiled into your kernel which will never need to be used.
I'm not sure this is true. Device drivers are typically compiled as modules, which are only loaded into the kernel if needed. Common wisdom among Linux power users for the last decade or so has been that compiling your own kernel for any tangible performance/memory improvement is basically futile.
Are there exceptions to this that you are aware of, and can elaborate on?
Writable caches yes. However many (most?) modern processors also contain an code-only cache which can be used for OS caching without any need to flush. The raspberry pi 4 can fit many modern microkernels in their tiny immutable cache, so it gains a massive caching benefit by being small.
> Are there exceptions to this that you are aware of, and can elaborate on?
You’re probably right, I was unaware of the extent of the kernel module usage. Regardless, the smallest common kernel in use that I know about is for alpine Linux and that is still at least 100MB, which is dramatically larger than the <1MB microkernels that are typically available.
OK then, so, which microkernel provides performance on-par with or better than Linux?
This is a strong claim and some evidence is warranted.
Syscall latency also is not the end-all-be-all of performance. Does that actually translate to faster disk and socket I/O on average?
Syscall latency is definitely not the end-all-be-all of performance, but it is all there is to compare between a monolithic kernel and a microkernel. In both cases, driver code needs to run, and that driver code can be good or shitty just like everything else.
When it comes down to it, the syscall latency that is comparable is the context switch between privileged and unprivileged mode. The holy grail of microkernels isn't to be faster than macrokernels, it is to be just as fast, but a hell of a lot more modular, usable, and secure.
That depends on what you're doing. If you're running some digital signage display or whatever, no one cares about a missing display frame or whatever. But if you're running a car's HUD or, worse, its steering system, you absolutely need the execution time guarantees or your code won't make it past certification.
This makes no sense. The comparison that matters isn't kernel vs microkernel, its kernel vs microkernel plus all the userland code needed to implement system services. If my program wants to read a file and write to the network, then the code to do that has to be brought to the CPU to run before my program can finish doing that, and it doesn't matter whether that is in the kernel or userland.
And also, what matters for performance is the size of the "hot" code. A lot of code, both on a monolithic kernel and on the userland code implementing system services for a microkernel, is going to be "cold" code which executes rarely. This includes initialization code, shutdown code, most of the error handling code, code to react to uncommon hardware events, and so on.
> If my program wants to read a file and write to the network, then the code to do that has to be brought to the CPU to run before my program can finish doing that, and it doesn't matter whether that is in the kernel or userland.
And given that the essential complexity is the same, it's very likely that the size of the relevant code is similar.
The reason microkernels typically benchmark on syscall (typically IPC) latency is because that is the only truly apples to oranges part that you can't compare between the two. Monolithic kernels have a single context switch, microkernels have two. But microkernels, being micro, are a lot friendlier to caches (less to load and flush), and especially so with dedicated code caches (which do not need to flush on context switch).
So even if you have a really large monokernal the parts of it that actually get used will be in cache in the parts that aren't used and won't be in cache.
Has other people stated how Michael Colonel is still going to need a lot of user modules to do most of the work that's normally in the monolithic kernel, so it'll probably end up being about the same amount of code that needs to be in cash first, not in cash. The differences is really come down to just call boundaries as I understand it
Too big for what exactly? Because it's clearly not too big to run. It's been running very well for decades, and Hurd has not.
I've known for decades that on paper, microkernels and Hurd should be superior, and I've wondered for nearly as long why then they aren't taking off. If they're so good, they should at least be thriving in some niche where that performance matters, shouldn't they?
And now I hear that even after 3 whopping decades, it still doesn't run on real hardware! Entire operating systems have been born and died in that time.
What is the problem that's making Hurd still not work?
> 2. It will take time to write a ton of device drivers, simply because most of them aren't written yet. To that extent, Linux most definitely has an incumbent advantage.
But why is Linux the incumbent when Mach and Hurd were conceived 3 years earlier?
> But that isn't to say that microkernels also have an advantage here.
I would love to believe that, but then why has that advantage not paid off for 30 years?
> With all that being said, I definitely don't think Hurd is the future. A Why not? And is this why nobody has been writing those drivers?
I don't mean to be shitting on Hurd, but it's been really promising for so long, and not delivering, that it's starting to feel like fusion power: always in the future, nearly there, but never quite here.
They (microkernels) thrive in Automobiles, Aircraft, Spacecraft, Security systems (such as hardware encryption devices), Defense Munitions, Embedded Devices, Maritime devices, radar systems, and many other areas. The common thread here is that they are not just resource constrained and performance sensitive, they are also typically very specialized and mostly write-from-scratch.
That last part is the key to understanding why general purpose operating systems mostly use legacy kernels that are decades old. We want to preserve as much as we can from what came before.
> But why is Linux the incumbent when Mach and Hurd were conceived 3 years earlier?
Because Linux worked earlier. Nobody in the GNU ecosystem knew how to develop a microkernel that worked, let alone drivers for a constantly changing target OS.
> I would love to believe that, but then why has that advantage not paid off for 30 years?
Again, they have. Almost all RTOSes are microkernels, and the most popular embedded OSes are microkernels. When you're writing something from scratch, it is always going to be easier with a microkernel. Don't mistake Hurd with the concept of a microkernel. Hurd sucks because it has always sucked. It's bloated, poorly documented, relies on a very janky GNU ecosystem that has already mostly adapted to Linux out of practicality concerns, and there is no ecosystem tooling around it.
But the advantages of microkernels in general are very real. The best one out there in terms of maturity is QNX, and while practically every modern car on the planet runs on it, it's licensing pretty much precludes it from being used as a general purpose OS.
But it's not being used there in current mainstream production.
https://www.meaa-mea.com/advanced-mobility/
https://www.blackberry.com/us/en/success-stories/mitsubishi
It's certainly difficult to pick through the Blackberry QNX hype to see where it's actually headed outside of infotainment, lot's of fuzzy weasal phrasing:
https://blackberry.qnx.com/en/industries/connected-autonomou...
Outside of some vague blackberry press materials, I've never seen anything specific.
Outside of some vague blackberry press materials, I've never seen anything indicating use in vehicles.
Or could they work if only people would stop focusing on Hurd? Is Hurd a dead end? Or is GNU the real problem and should we abandon that?
Out of all of the microkernels out there, the one I like the most is SEL4, but the security model is so different from most OS kernels that you essentially have to program for it from the ground up. And it seems like there isn’t any momentum there either. QNX is already very mature and performant though, and it has gotten a huge boost from IoT and electric car development, and IMO all it needs is an open source license and it will eventually creep its way into general purpose OS usage.
Yes, it's old. Yes, there's only little interest. Yes, there are _still_ many technical problems.
The interesting thing here is that technical problems can be solved, and will be solved given enough time. Heck, I've heard rumors that even Windows does not crash multiple times a day anymore.
Focussing and working on what is popular will most likely create a world that is less free every year. Some people rather work on something that seems right and promises a better, freer future. Even if it's not popular (right now), even if it has a high probability of failure.
Popularity is a very volatile property...
And popularity might be a double edged sword, but double edged swords are still useful if you're careful.
Although keeping software proprietary or writing proprietary software may have become somewhat unpopular in some niches (I'm not sure it's incredible unpopular seeing how many people and institutions still use and often advocate Apple, MicroSoft, use FaceBook, WhatsApp, whatsnot...), I'm not yet convinced that many people (say more than 1%) really believe that all software should be free and act on that?
These days most Linux distros build pretty much any driver as a module that can be built as one. Boot-critical modules (like NVMe drivers) are included in the initramfs so they're available before the disk is mounted.
I agree that Linux is still larger than a microkernel, but very few users have a bloated kernel in RAM due to lack of customizations.
Most users run the distribution's kernel.
The rest is in the modules and they're not loaded unless they're actually used.
What could possibly go wrong.
So yeah, in the ideal world, let's have everything isolated. For now it would be great if they worked at all.
Also... There's a bit of practical limitation to how useful the isolation is. It's cool that the FS module runs in the userspace. But if it can create a suid-equivalent file and point an arbitrary config at it, you're not gaining much.
I am not going to pretend I have any idea because I have just googled these numbers but it seems to me that an amount of cache could be reserved for the kernel without too much of a hit to applications on what are currently high end machines.
1. Linux is too big. Even if we exclude the drivers, which are a majority of the kernel codebase, linux is still a massive kernel. Most microkernels are small enough to fit in the L3 cache, some are small enough to fit in the L2 cache. Linux, even if we could exclude the drivers, doesn't even come close. This inability to fully cache the kernel ends up negating the primary benefit that monolithic kernels have, which is performance. Linux spends more time bouncing around the cache than a modern microkernel takes to do an extra syscall.
To add to this, unless you are compiling your kernel specifically for your machine, your kernel is going to be a bloated compromise of the set of drivers that your system is most likely to see. That means that you will have dozens, if not hundreds of drivers compiled into your kernel which will never need to be used.
It's also too big in the sense that there are now far too many lines of code to be able to effectively monitor for vulnerabilites. That codebase size won't change when having a microkernel with the same number of drivers, but what will change is the ability of a single vulnerability to compromise your whole system. You have effectively compartmentalized vulnerabilities to their own address space. So in that sense, big codebases are a huge problem for monolithic kernels, but only a small problem for microkernels.
2. It will take time to write a ton of device drivers, simply because most of them aren't written yet. To that extent, Linux most definitely has an incumbent advantage. But that isn't to say that microkernels also have an advantage here. Anyone who has written a driver knows how much the ability to iterate quickly is helpful. Drivers in userspace can be written directly on the system that they will run on, with no virtualization or kernel reboots necessary, and with incredibly fast start/restart/stop. Userspace drivers are dramatically simpler to write, simpler to build, simpler to monitor, simpler to test, and simpler to distribute.
With all that being said, I definitely don't think Hurd is the future. But what could change the world is already out there but not really ready to use at the moment. One possibility would be an open-sourcing of QNX which is an amazing and incredibly mature microkernel OS, possibly the most mature in the world. If there are any bored billionaires out there reading this, please buy QNX from Blackberry and do that. Another possibility would be to put a shit ton of effort behind the SeL4 userspace, which is just too damn hard to do anything practical with at the moment.