The big missing "virtualization" technology is the Apache/CGI model. You essentially upload individual script-language (or compiled on the spot) functions that are then executed on the server in the context of the host process directly.

This exploits the fact that one webserver only differs from another by the contents of it's response method, and other differences are actually unwanted. You can make this a lot more efficient by simply having everything except the contents of the response method be shared between different customers.

This meant that all the Apache mod_x (famously mod_php and mod_perl) can manage websites on behalf of large amounts of customers on extremely limited hardware.

It does provide for a challenging security environment. That can be improved when starting from scratch though.

I think the modern equivalent of what you are describing is basically the AWS Lambda model of "serverless" applications. In the open source world, there are projects like Funktion[1] and IronFunctions[2] for Kubernetes

[1] https://github.com/funktionio/funktion

[2] https://github.com/iron-io/functions

I get that that's what they're saying, but it just isn't. Functions are just a way to start containers on an as-needed basis, then shut them down when not needed.

Mod_php is 3 syscalls and a function call, and can be less if the cache is warm. Despite the claims on that page, there is no comparison in performance.

"Extremely efficient use of resources"

It is utterly baffling that one would use those words to describe spinning up either a container or a VM to run these lines of code (their example), and nothing else:

  p := &Person{Name: "World"}
  json.NewDecoder(os.Stdin).Decode(p)
  fmt.Printf("Hello %v!", p.Name)

Number of syscalls it needs to switch into this code ... I don't know. I'd say between 1e5 and 1e8 or so. Probably needs to start bash (as in exec() bash) a number of times, probably in the 3 digits or so.

So I guess my issue is that functions use $massive_ton_of_resources (obviously the lines of code printed above here need their own private linker loaded in memory, don't you agree ? It's not even used for the statically linked binaries, but it's there anyway. Running init scripts of a linux system from scratch ... yep ... I can see how that's completely necessary), but when they're not called for long enough, that goes to 0, at the cost of needing $even_more_massive_fuckton_of_resources the next time it's called.

Of course, for Amazon this is great. They're not paying for it, and taking a nice margin (apparently about 80%, according to some articles) when other people do pay for it.

And the really sick portion is that if you look at how you're supposed to develop these functions, what does one do ? Well you have this binary running "around" your app, that constantly checks if you've changed the source code. If you have, it kills your app (erasing any internal state it has, so it needs to tolerate that), and then restarts the app for the next request. Euhm ... what was the criticism of mod_perl/mod_php again ? Yes, that it did exactly that.

A container needs 10-100 syscalls, depending how much isolation you want. A single unshare() and exec gets you some benefit. You are out by orders of magnitude.

And then of course the system inside the container needs to start up, configure, run init scripts, ... Did you count that in those 100 syscalls ?

Take the example here: https://github.com/kstaken/dockerfile-examples/blob/master/n...

Which does something a lot of these functions will do : get nodejs, use it to run a function. Just the apt-get update, on my machine just those instructions, ignoring actually running the function (because it's insignificant) does close to 1e6 syscalls.

Lightweight application containers do not run init or anything like that! They're just chroots but with isolated networking, PIDs, UIDs, whatever.

For example, on my FreeBSD boxes, I have runit services that are basically this:

exec jail -c path='/j/postgres' … command='/usr/local/bin/postgres'

Pretty much the same as directly running /usr/local/bin/postgres except the `jail` program will chroot and set a jail ID in the process table before exec()'ing postgres. No init scripts, no shells, nothing.

I don't understand the criticism. FreeBSD jail is more like chroot than like a container. A container, as I understand it, runs it's own userland. Otherwise, you can't really isolate programs in it. If that postgres was compiled with a libc different from the one on the host system, or let's say required a few libraries that aren't on the host system, for instance, would it run ?

Does it have it's own filesystem that can migrate along with the program ? Does it have it's own IP that can stay the same if it's on another machine ?

You're correct. Containers do contain their own userlands, a fact many gloss over. PgSQL will have to load its containerized version of all libraries instead of using any shared libraries linked by the outside system.

This is often done via a super thin distribution like Alpine Linux to keep image size down, despite the COW functionality touted by Docker that's supposed to make it cheap to share layers.

The difference is that unlike a fully virtualized system, the container does not have to execute a full boot/init process; it executes only the process you request from within the container's image. Of course, one could request a process that starts many subservient services within the container, though that is typically considered bad form.

What people really want is super cheap VMs, but they're fooling themselves into believing they want containers, and pretending that containers are a magic bullet with no tradeoffs. It's scary times.

Even a basic chroot runs its own userland! "Userland" is just files.

In my example, /j/postgres is that filesystem that can migrate anywhere. (What's actually started is /j/postgres/usr/local/bin/postgres.) Yeah, you can just specify the IP address when starting it.

What system? Your link just starts a nodejs binary, no init process. And you also don't seem to realise that a docker container is built only once? Executing apt happens when building the image (and then is cached for if a rebuild happens later), not when starting it.

These steps are only run for initial creation of the container image. Running the container itself is only the last step from that file: Executing the node binary.

I am not quite sure what it is that you want. It seems obvious to me that containers should have more overhead than CGI scripts; it also provides a better isolation story. I mean, you already said it:

> [the Apache/CGI model] does provide for a challenging security environment. That can be improved when starting from scratch though.

And the number of lines of code in the example probably doesn't quite matter so much, because that's all it is: an example. I am sure that you can run more lines of code than that.

> Euhm ... what was the criticism of mod_perl/mod_php again? Yes, that it did exactly that.

I mean, that's also basically my point, that Lambda is basically the CGI of the container world. Lambda and CGI scripts really do seem like they are basically the same thing; I still speculate that they will be used to fill similar use cases. I am not really opining on which one is actually better.

You can share resources between VMs (frontswap etc. and deduplication, using network file systems like V9FS instead of partitions) but it complicates security.

It is still safer than containers as one kernel local root bug does not break a VM, but breaks a container. The access to hardware support also allows compartmentalized drivers and hardware.