> there's little need to SSH into the system in the first place (except for debugging purposes).

Pretty big except though.

This concept has been done multiple times and has always failed. It's solving a problem that few care about.

> It's solving a problem that few care about.

Security?

exactly this, debugging is a pretty big part when it comes to operational aspects.

but I feel like similar to kubernetes container injection systems. I suspect in future we would see dynamic injection into the unikernel where you can enable debugging environment instantly.

like the library unikernel exposes a plugin system that you can inject in future, this way the unikernel is lightweight and only gets bloated when you eventually want to debug something.

actually I think this might be the intention of mirage as well, especially since they put "modularity" in the title

What you already can do is attaching gdb -- either live or at a core dump. Please see https://github.com/Solo5/solo5/blob/master/docs/debugging.md for detailed instructions ;)

What you as well can do is record-replay -- well, at least there used to be this option, I'm now lost whether it has never been merged anywhere, or it is stuck in some PR somewhere. This was truly great - since the external interface is so thin, it is easily doable to dump all external events (API calls and returns) onto disk and replay one-by-one, inspecting the state.

In the kind of corporate environments I work on, you won't be doing any SSH into the production containers.

Nothing that a classical UNIX admin would expect is installed on those images.

Yes. Welcome to the hell of "No debugging, no fixing anything. Call vendor support and hope for the best"

It is more like, what isn't there can't be used for CVEs.

Those kind of corporate environments would also never use a technology like MirageOS that very few people know how to use.

In those kind of corporate environments liability is a concern, and something like MirageOS would be quite appealing.

These are the same kind of environments where CI/CD can only fetch from internal repos, where stuff only gets installed after IT and legal had a say on it.

Sure they would. I've be interested in Mirage and I don't believe that developers should have SSH access to systems.

I could see the value in a debug port or something, though. But giving full remote code execution is total overkill.

... we had to wait for k8s 1.25 to get ephemeral containers to get some tools ... (https://kubernetes.io/docs/tasks/debug/debug-application/deb...)

I haven’t SSH’d into a machine/container to debug or fix a production issue in years. You flat out can’t even SSH into anything on the last few prod setups I’ve worked on: permissions prevent it, and most containers have sshd and friends stripped out.

I would love a super slimmed down stack to run workloads on. Most of the time when I run workloads, I want to run the workload and nothing else. The OS in the container exists to start the binary I care about and then basically very little else-these applications aren’t making use of a scheduler, or the init system (because again, I’m only running my binary), users and permissions go unused a good chunk of the time. With stuff like quic becoming more popular we’ve got user space networking too, so that’s out. So by this stage we’ve got a whole OSCthats doing almost nothing except passing stuff between layers an incurring a performance and risk overhead.

If we could have the same functionality, with better performance and the same or better security, why wouldn’t I do that?

> This concept has been done multiple times and has always failed

Isn’t it likely that this idea was just “before its time”? The compute landscape these days is pretty different to ~20+ years ago: massively available public cloud compute, prolific container usage, etc. Software stacks are deeper, more ossified and less fully understood than ever, I definitely see solutions that reduce that complexity being appealing.

What schedules your threads?

But what's the benefit of cutting out the full-blown operating system? It's rare that such things would be the bottleneck.

Performance and security, basically.

Similar to trying to run containers bare metal directly on top of type 1 hypervisors, except unikernels are designed for this purpose.

At least for cloud deployments you don't need a full blown OS when all you want to do is run apps like NGINX or Redis. If you use a unikernel instead, then you severely (orders of magnitude) reduce cold boot times, memory consumption, server density (thousands on hw-isolated instances on a single server) and TCB.

Makes sense. Thank you.

Isn’t that the aim of webassembly?

Not really, as the filesystem you access from Webassembly needs to be in a sort of sandbox or container too.

Not at all, WebAssembly is replacement of PNaCL browser vendors could agree on.

Then a bunch of folks, with some VC money, decided to make the second coming of Java and .NET with it.

Isn’t this the same idea behind unikernels?

Mirage is a unikernel.

The catch here is Ocaml. You can design your unikernel "bare metal" in any way you want and make it do nothing but the essentials you need ... and design it in any language you want as long as that language is OCaml.

And on that note, I just found this list of UniKernel projects:

http://unikernel.org/projects/

I have especially had hopes for the UniK [1] project, as it was/is written in Go AFAIK. I see now it incorporates work from the Mirage project as well. Not sure what is the status of this project anymore though.

[1] https://github.com/solo-io/unik

I didn't read your message, so I just clicked and I found that great nice project from Solo.io (the company behind Gloo Edge, etc.). Freaking awesome... I thought:

> Not sure what is the status of this project anymore

what the hell! :D last commit was 4 years ago!

Hah, yea, should have checked that :D ... but yea, I stopped hearing news from solo.io / UniK some time ago, so it makes sense.

Unik was just a build tool that utilized other projects like Rump, Mirage, IncludeOS, etc. It's now dead since Solo pivoted a very long time ago to service mesh/api gateways.

The GoRump port they use was from us and then we realized we needed to code our own from the ground up for many reasons so we wrote https://nanos.org (runs as a go unikernel in GCP).

I tried to search if there are easy ways to do bindings to other languages. There seems to be bindings to C which should make it easier to do bindings to other languages too, but I am not sure whether someone has done that work just yet. Until then, I really doubt this thing will fly. Great project, nothing to say, but OCaml, ...

For a language-agnostic unikernel project you can also check out www.unikraft.org (OSS/Linux Foundation, actively maintained).

That’s what this is…

> MirageOS is a library operating system that constructs unikernels for secure, high-performance network applications across a variety of cloud computing and mobile platforms.

Absolutely, MirageOS is indeed pivotal in the development of unikernels. MirageOS not only developed the unikernel concept but also coined the term 'unikernel'.

MirageOS represents a significant shift in cloud and network computing, focusing on building highly specialized, secure, and efficient unikernels that streamline application deployment by shedding unnecessary components. It also puts extra focus on security, with most of the stack — including TCP and TLS — being rewritten in a type-safe language (OCaml).

Currently, MirageOS is expanding to include bare-metal and embedded systems, aiming to bring the same level of security and efficiency to this domain. The move towards embedded systems is a natural progression for MirageOS, given our emphasis on minimalism and security in environments where resources are limited and reliability is crucial. See for instance what is happening with SpaceOS: https://tarides.com/blog/2023-07-31-ocaml-in-space-welcome-s...

> I'm really sold on the idea: Instead of a full-blown OS,

You mean you are sold on the idea of going back to the 80s. We used to have an operating system with every game released[1].

1. https://youtu.be/kZRE7HIO3vk?t=1114

A big difference in my opinion is that projects like Mirage does not try to reimplement the same hardware drivers as the host system and instead implements much simpler drivers for virtual hardware.

For example, how do you write to a block device? You make a hypercall with a "handle", offset into the block device, the number of blocks to write and a pointer to memory to be written. There's no pretending it's a spinny disk with sectors and heads or what have you.

https://github.com/Solo5/solo5/blob/bf29b8af11feec9dbc2e74cc...

That is just infinitely worse than calling write() on a file handle like a regular application.

The hypervisor + OS + application model is stupid. The hypervisor and OS are duplicative. If you can rewrite the application (as is needed for a unikernel design) there is no reason to use a hypervisor in your stack at all. You should just be running a OS directly on the bare metal with no hypervisor at all.

The only reason to prefer a unikernel is because you wrongly believe that hypervisors are a security boundary. Everything else about them is strictly inferior to a standard application on OS model.

One of the key insights for unikernels in general is that everything is virtualized now to begin with. The entire public cloud: AWS, GCP, Azure, etc. is virtualization with an api on top of it.

So unikernels take advantage of this fact to get the performance, security and ease of use benefits.

Hypervisors most definitely have better hardware-based isolation than a few processes running in linux. In fact the isolation is so good that the entire public clouds are built on this model.

This only works if your environment is bare metal to begin with. In some cases you may be in an environment where a lot of applications are proprietary, virtualized, or otherwise constrained in some other way. It is not an environment where you get to pick and choose how everything is setup. In such a case, having a distinct small application that fits as "an OS" in a virtualized environment can fit a lot better in, than trying to retrofit a baremetal containerized environment into a large enterprise datacenter where everything is already running Windows servers and proprietary services, managed by other people than yourself.

>The only reason to prefer a unikernel is because you wrongly believe that hypervisors are a security boundary.

Thoughts on Qubes?

> The only reason to prefer a unikernel is because you wrongly believe that hypervisors are a security boundary.

That's wrong. They rightly believe that traditional operating systems deployments come with much larger attack surfaces.

There is no meaningful security difference between commercial operating systems and hypervisors when considering technically competent, commercially-motivated attackers. No systems built on such insecure systems can stop even minor attacks staffed by a single-digit number of FTEs. Maybe there is a material difference, but it is irrelevant compared to their total inadequacy relative to the present threat landscape. The person who can climb Mount Everest is still no closer to achieving orbit than the person who can only climb a tree.

To move beyond the mere practical aspects, even theoretically you are wrong. The techniques needed to develop a secure hypervisor are basically exactly the same techniques needed to develop a secure operating system. They are almost trivially transferrable. If you can do one, you can do the other. So, again, no advantage to preferring a hypervisor based solution.

Even if I accepted the claim that the techniques are the same, it's irrelevant. It's simply a matter of fact that standard installs of commodity server operating systems have a significantly larger attack surface than a unikernel system. Sys loggers, email agents, inflexible and insecure user/group access control, and just general ambient authority. Unikernel systems have none of this cruft by default.

Well it doesn't make much sense with games and other user applications nowadays, but if you want efficiency and a smaller attack surface in a production environment, why not? People already use Docker containers that were stripped to the bare minimum for the same reasons.

Well, technology is known to spiral around ideas with time, so nothing's wrong with that

I think many designs in tech have two states between which they oscillate. We don't go back exactly to the same thing, we do improve on things, but things are very similar to the way they were done 20-30 years ago (or 40 in this case)

Well, that is kind of what the maker community is doing with Arduino and ESP32, and what makes them much more interesting to hack around, as yet another UNIX clone.

To be a bit more precise, what we achieved so far: apart from web servers and TLS reverse proxies (with DNS-backed let's encrypt provisioning), DNS authoritative servers, a CalDAV server, git client and servers, SMTP stack (including dkim etc.), OpenVPN implementation, archive mirror (using tar as file system), monitoring with syslog and influx, ...

A big milestone was binary releases and deployment instructions, see https://robur.coop/Projects/Reproducible_builds

see more at https://blog.robur.coop https://hannes.robur.coop https://blog.osau.re/ https://reyn.ir/archive.html

If you're curious how/what to integrate MirageOS into your infrastructure, please reach out (best via email).

> I'm really sold on the idea: Instead of a full-blown OS, you compile your application with a thin layer of support libraries that provide the OS features that your application needs

I'm really sad that unikernels never took off, even there seemed to be a lot of excitement around them a few years back.

The second wave is coming :) . At www.unikraft.org we're hard at work providing a Linux-API compatible unikernel (run unmodified apps/langs) and combining it with tooling like docker and k8s.