This thread was very helpful:

> Explain this to me then: What's the difference, conceptually, between (A) a VM host running a unikernel capable of solving one particular kind of problem, and (B) an OS running on bare metal running a process capable of solving one particular kind of problem?

They are very similar. The main difference is the shape of the OS APIs. In the former scenario, the host provides things that look like CPUs, NICs and things that look like block devices, with no high-level abstractions (e.g. threads, sockets, and filesystems). If these are needed, then the unikernel will link in device libraries that provide them. In contrast, in the later situation the OS will be providing these even when they are not needed and they will add overhead to those code paths.

Additionally, the amount of process state owned by the OS is far greater in the latter case. Things like file descriptors, thread priorities and so on make it far harder to snapshot or migrate an OS process. Hypervisor interfaces tend to be as close to stateless as possible.

> In the former scenario, the host provides things that look like CPUs, NICs and things that look like block devices, with no high-level abstractions (e.g. threads, sockets, and filesystems).

To add to this, with the approach we're taking with Solo5 [1], hardware virtualization is only used as one possible sandbox/isolation mechanism. If you look into the code, you'll find that the abstractions presented to the unikernel by the hvt tender are much thinner than those presented to a guest OS by a traditional VMM (e.g. KVM/QEMU).

[1] https://github.com/Solo5/solo5

I agree. One of the appeals of unikernels in my opinion is that (I think) they will eventually give us the things that Docker currently gives us, but with one less layer of abstraction (unikernels running on a VM orchestrator instead of containers running on a container orchestrator running on a VM orchestrator). I imagine that this is inevitable as a matter of efficiency, but it's probably such a small practical gain that we won't see much traction in the next decade.

> I imagine that this is inevitable as a matter of efficiency, but it's probably such a small practical gain that we won't see much traction in the next decade.

I agree that if the advantages boil down to saving a hundred MB of RAM of two, and enabling additional compiler optimisations, it won't be compelling to that many folks.

Taken to the extreme though, we can start spinning up instances 'just in time' to deal with incoming requests. Doing this, you could run (and hopefully pay for) your instances for only a tiny fraction of the time your service is in a working/available state.

This has apparently already been demonstrated [0], but their demo has been broken for as long as I can remember. I presume it worked at one point. I don't know if any commercial Xen-based offerings can spin up an instance as quickly as their configuration did, but the idea is there.

[0] http://erlangonxen.org/zerg

My VM is lighter and safer than your container:

https://news.ycombinator.com/item?id=15610155

Unfortunately the tooling story is poor for VMs. There isn't a good analog for the Docker toolchain (build, pull, push, etc) or container orchestrators (e.g., Kubernetes). These stories are at least as important as "lighter and safer".

I'd disagree here. The entire public cloud model is built on top of VMs. While I have a strong distaste for most CFM tooling out there that is precisely what you're talking about. Chef, Puppet, Salt Stack, Terraform, etc. all fulfill this and all of these are popular in their own communities.

I don't doubt that there are analogs for individual components, but I don't think there is anything comparable to Kubernetes for VMs. I also doubt the quality of those individual components or their integrations (for example, how do I build a VM image in <1m, what image repo technology should I use, what orchestration technologies exist and how do I tell it how to pull from my image repo, can I run that orchestration tech locally or on prem, etc?). I don't think these are insurmountable problems; there just isn't the same amount of investment (or rather, the investment is comparatively poorly coordinated).

LinuxKit gives build push run for VMs. It was kind if designed to be a half way stage to a full unikernel model.

I've played around with LinuxKit; it's quite difficult and limited. I couldn't get anything off the ground. Promising for sure, but needs more investment before it's ready for general purpose use.

I think container orchestration offerings are pretty much at the pricing model you describe. You can spin them up just in time and only pay for what you use. The advantage to unikernels is that cloud operators don't have to operate two layers of orchestrators, so they could pass those savings onto unikernel users; however, I don't think that will be the lowest hanging fruit for a long time.

I'd argue the opposite, which is that the future will have even more layers of abstraction, not less, since that's been the trend for quite some time.

My prediction is absolutely not "fewer total layers of abstraction", but only that this one will be collapsed in time. You can collapse one layer of abstraction and add two more, which is sufficient for my prediction to be compatible with the "more total layers of abstraction" prediction. :)

Or you could just run containers without the abhorrent mess that is hardware virtualization...

The post discusses running unikernels without "the abhorrent mess that is hardware virtualization".