Check out kraft.cloud and the accompanying LF OSS project www.unikraft.org :) (disclaimer: I'm one of the authors of the paper and one of the people behind that cloud offering). On KraftCloud we use Dockerfiles so users can conveniently specify the app/filesystem, and then at deploy time transparently convert that to a unikernel (specialized VMs). With this in place, NGINX cold starts in 20 millis, and even heavier apps/frameworks like Spring Boot in < 300 millis (and we have a number of tech to bring these numbers even further down).

Looks very cool.

For anyone else wondering how heavy this is on a MacOS, I ran the install script and it just delegated to brew... brew listed the following packages being installed:

    ==> Fetching dependencies for unikraft/cli/kraftkit: aarch64-elf-binutils, gmp, mpfr, aarch64-elf-gcc, coreutils, gettext, readline, gawk, gnu-sed, pcre2, grep, make, capstone, dtc, mpdecimal, ca-certificates, openssl@3, sqlite, [email protected], glib, libunistring, libidn2, p11-kit, libnghttp2, unbound, gnutls, jpeg-turbo, libslirp, libssh, libusb, ncurses, snappy, vde, qemu, socat, wget, x86_64-elf-binutils and x86_64-elf-gcc

Most should already exist on your mac if you do development... it seems to rely on qemu, unsurprisingly... openjdk as well (probably to support Java out-of-the-box?), imagegick etc.

Took a few minutes to finish installing... the CLI seems to be based on the Docker commands (build, clean, run, 'net create', inspect etc.), some package-manager like commands ('pkg info', 'pkg pull', 'pkg list' etc.), a bunch of "cloud" commands (I suppose that's the non-free part) and "compose" commands just like docker-compose. Interesting stuff.

Note for the parent commenter: the Lua link in the landing page is broken: https://github.com/unikraft/catalog/tree/main/examples/http-...

I tried to run the C hello world example... I get an error, it wants to run Docker?!?! I thought the whole point was to avoid Docker (and containers)??

Here's the log:

i creating ephemeral buildkit container W could not connect to BuildKit client '' is BuildKit running? W W By default, KraftKit will look for a native install which W is located at /run/buildkit/buildkit.sock. Alternatively, you W can run BuildKit in a container (recommended for macOS users) W which you can do by running: W W docker run --rm -d --name buildkit --privileged moby/buildkit:latest W export KRAFTKIT_BUILDKIT_HOST=docker-container://buildkit W W For more usage instructions visit: https://unikraft.org/buildkit W E creating buildkit container: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?: failed to create container

PS. running the hello-world pre-built "image" worked:

> kraft run unikraft.org/helloworld:latest

EDIT:

A lot of stuff looks broken on MacOS.

For example, `kraft menu` doesn't work (error "no choices provided", even though the docs show it working fine without "choices"?)...

`kraft run --elfloader loaders.unikraft.org/strace:latest ./my_binary` also doesn't work (the docs show it working). Error: "unknown flag: --elfloader".

Seems like the product is still in alpha?!

Is the entire kraft stack open source? Can I run this on my own hardware?

Also was my question.

VMs in general use more CPU power as you have two OSes each doing things like updating their real time clock... There are VM aware OSes that will not do this, but it needs special code and CPU support which means you are often lagging behind the latest (to be fair this is rarely important) A container will normally be slightly faster than a VM never slower (assuming a reasonable OS - I can write an exception if I was malicious) and so there is a lot of interest if they are good enough.

At the time of publication of the article the tool used to create the minimalistic VM Tinyx was not released and as far as I can see was never released.

Correct, we never did release Tinyx, mostly because it was in a very unclean/researchy state = not ready for public consumption. In retrospect, we probably should have either (a) made it available in whatever state it was in or (b) put more cycles into it.

But then also monitoring gets harder. With containers you could see (and monitor) all those processes running in container easy on the host.

With VMs? You now need a way to get data from within the VM which in most lightweight VM implementations just not possible.

On kraft.cloud we have unikernels (specialized VMs) with Prometheus exporters that can be scraped, and other monitoring facilities.

I wouldn't discount the massive user base of containers among developers.

i think at that point it's kind of just semantics

i don't think devs care if they use containers or VMs, as long as it's easy and they don't have to worry about which version of Python the host is running

This. It's why vagrant was popular before the container revolution.

The killer app of Docker isn't the container, it's the depth and uniformity of the UX surrounding the container system. When that is broken by something on the host (non x86 cpu was a major pain for a while before popular images were x-built) and emulation gets in the way and is not as easy, or just mildly different (windows behind corporate firewalls that assign ips used by the docker engine for example), the ease of use falls away for non-power users and it's all painful again.

Tech like Docker for windows and Rancher Desktop and lima has largely matured at this point, but somebody could make a new machine and then the process of gradual improvement starts all over again.

No it didn't. I want to rebuild my image using a different base.

Docker? One line possibly.

VM? Afternoon (unless I want to write ansible too)

In this sort of scenario, then you can also use something like cloud-init: https://cloudinit.readthedocs.io/en/latest/

Or I could use the tool I currently am. Your reply doesn't solve my use case it just says use yet another tool.

You don't need a VM on Mac to run containers, check out OrbStack, they provide a Docker compatible engine that is using native MacOS capabilities for running containers without the hidden Linux VM.

I don't know where you got that idea. OrbStack absolutely runs a Linux VM. That Linux VM then uses Linux containerization technologies (namely LXD) for each separate OrbStack 'machine' you set up, which is how you get such fast startup times for your OrbStack 'machines'.

For Docker, OrbStack does the same thing as Docker Desktop, Podman Desktop, Rancher Desktop, etc., which is set up a Linux VM running Docker and then present a native socket interface on macOS which relays everything it receives to the Docker socket inside the VM.

macOS doesn't have native capabilities for running containers, which is why the nearest thing you can get to containerd on it requires you to disable SIP so it can use a custom filesystem to emulate bind mounts/null mounts: https://darwin-containers.github.io/

If you read the PRs where the principal author of the Darwin Containers implementation is trying to upstream bits of his work, you'll see containerd comparing his approaches to others and complimenting them by calling them 'the most containerish' because real capabilities aren't there.

(I believe I've read rumors here on HN that Apple has those features internally, fwiw. But they've evidently never released them in a public copy of macOS.)

Another clue in all this is to just run uname in any of your Docker containers in OrbStack; you'll see they're Linux machines. Some operating systems have Linux syscall emulation layers (WSL1, FreeBSD's Linux emulation, Illumos' LX Zones) that could perhaps be used to run Linux containers without hardware emulation or paravirtualization in combination with some native containerization capabilities. Afaik Illumos' LX Zones is the only implementation where that's a supported, intended use case but maybe FreeBSD can do it. At any rate, macOS has never had that kind of syscall compatibility layer for Linux, either. So when you run `uname` in a 'macOS container' and see 'Linux', you can be certain that there's a VM in that stack.

PS: Aside from the fact that it's proprietary, I really do quite like OrbStack. It's the nicest-to-use implementation of something like this that I've tried, including WSL2 and Lima. The fact that it makes the VM machinery so invisible is very much to its credit from a UX perspective!

Interesting! I'd swear that in the early days of OrbStack somewhere on their website I've read they're using native MacOS frameworks without the need of Linux VM, but I can't find that anymore (they don't mention Linux VM either, but the language still differs from what I remember).

They do use native GUI frameworks rather than something like Electron, which they still mention. And maybe they also used to have something about relying on Apple's Virtualization Framework or something like that, rather than qemu as Lima used for a long time. (I think it may still be Lima's default, but not for long.)

Certainly depends a lot on what the term "VM" actually means in the context. If it's something as specialized as the JVM, or a native virtualization with an extremely trimmed down guest, then at some point you'll find yourself in need of something more heterogenous, e.g. running a tool on the side that does not fit the VM. Then you're back at square one, only this time with containers (or back in some irreproducible ad-hoc setup). Going with containers from the start, containers that may or may not contain a VM, and that may or may not actually do more than what a VM could supply, that's much less hassle than changing horses at a later point.

Where is the tooling to build and distribute lightweight vms like containers? How can I copy one html file into an nginx VM, built this vm image with multiple architectures (I have arm and x64 servers), publish it, pull it, and run it multiple times?

Once again: Containers are not about isolation or security, they are a package format for shipping applications. The packages are easy to build, distribute, multiarch, ...

And requiring a Linux-VM on macOS to run Linux containers, is not particularly surprising.

> specially given that on my Mac I need to use a VM anyway to run containers

Take it up with Apple.

I agree in principle, but this is not so easy when shared disk IO is introduced, in my experience.