Shutting them down right away achieves a number of things, such as significantly better isolation between requests, and more protection against resource leaks.

Whether that'll be worth the cost remains to be seen - 250ms is still far too long for most uses, but if it can come down enough, it's attractive.

I couldn't get to the site, but assuming this is running Linux I wrote a paper about how fast you could boot a Linux minimal userspace on QEMU a couple of years ago[1]. In short if you're willing to heavily customize the kernel, qemu and userspace then you can get it down to 100-150ms on i7-type hardware, but probably not much lower. Using a distro kernel, distro qemu and so on, you're stuck with 400-600ms (which is fine for certain uses, eg better security for containers like Kata Containers is doing).

Also worth noting I wrote a bunch of benchmarking tools to help optimize this case[2].

[1] http://oirase.annexia.org/tmp/paper.pdf

[2] https://github.com/libguestfs/libguestfs/tree/master/utils

Thanks for the links. I've looked at Clear Containers/Hyper/Kata before, and it's absolutely looking good. It's very interesting to see your work on going into details on it, though.

I really hope this can be pushed further down to the point where it gets viable to for more use cases. Of course, we can always use something lighter than Linux, but being able to start with Linux and pare down is potentially so much easier for many types of applications.

> own serverless implementation

Isn't this just cgi-bin, right down to the long startup time and lack of persistence?

It's similar in intention, but "serverless is just `cgi-bin`" misses a lot of nuance. Zerg and cgi-bin are solving different problems and with different implementations. It's extremely difficult, if not impossible in some environments, to get the same kind of isolation with cgi-bin that you do out of a VM.

You gain on isolation with Zerg (and other "VM per request" things) but you lose on everything else compared to CGI with processes.

But you can reach comparable levels of isolation with processes: just use Linux namespaces.

Is it? I'm under the impression that cgi-bin is a way to serve requests from the local computer, while Zerg is a way to spawn _other_ server instances to handle the request (and in Erlang, specifically).

Yes, but you could let a load balancer handle that for you, and you might not need as many computers to handle the requests if it didn't take 250ms to start.

Booting a VM should be "as complex" as fork. And the memory image of the booted OS should be identical on each load before the end user specific data is loaded.

"Prefork" these VMs by spraying that image over physical memory in advance, jump to them on a request. memcopy a new VM over them on exit.

VM spawn rate should be the memcopy rate, if the MMU is powerful enough, should only be the amount of memory mutated during a request.

I have written a few web services in the past where this kind of isolation would have been great. Services doing file conversion of untrusted input with software that isn't hardened for that case (in my case converting PDF to png and postscript).

The point is that you don't need the added complexity of a worker pool. It's another thing to break, so why bother?

Even 250ms latency to boot is significant. Especially if you're processing thousands of requests per second.

Or if you are processing more than 4 per core per second.

Threads take milliseconds to start yet we still use thread pools.