AWS Lambda in particular will spin up a new container running their framework, which will then stay alive for a period of time before being terminated.
With socket activation (using xinitd or systemd) and a minimal framework, you could implement much the same on your own VMs.
That's true. I should have pointed out that from the developer's point of view it appears as if the execution is terminated, in the sense that no state is carried across code executions. However, the issue of whether the code is cold booted or not is an implementation detail of a specific serverless platform.
Atlassian did a nice blog post[1] on running Docker containers using socket activation. In principle that makes for a serverless platform but for a one that's difficult to manage. [1] https://developer.atlassian.com/blog/2015/03/docker-systemd-...
If you can't maintain state inside the application, you need to maintain it elsewhere (DB, Redis, memcache), and that elsewhere has a cost associated with accessing it.
Consider session state in a web app. If you keep state in memory and scale horizontally, you need session affinity, so stateless inbound http connections hit the same server instance. That means a session-aware router, delicate and extremely expensive F5s and other "smart" routers. But if the app is stateless and session state is in external storage, then all you need for load balancing is a simple round robin. A cheap router, that's more reliable and requires less administration, for a fraction of the cost.
There are other ways stateful apps are expensive, but that's a big one. I'd rather pay $5k than $100k for a router.
However, it'll be in the middle of the range for startup times, as some agents are sideloaded. Latency is around a second. The initial target is tasks like database migrations and batch jobs, so the ~1s delay isn't an issue.
One of my colleagues in NYC spiked on using one-off tasks for a FaaS and concluded it's not a slam dunk fit.
If I was to shimmy FaaS-style stuff into Cloud Foundry, I think it'd be easier to have standing apps with an array of functions already running, then push the magic into route services. I suspect this is how AWS already does it, given the vast difference in spinup times between Node and JVM requests -- Node makes hot-patching a list of functions easy, the JVM is less hot-patch friendly and really expects you to provide all the JARs at launch time.
Disclaimer: I work for Pivotal, we donate the majority of engineering to Cloud Foundry.
[1] http://0pointer.de/blog/projects/socket-activated-containers...
[2] https://developer.atlassian.com/blog/2015/03/docker-systemd-...
Obviously this is tightly locked down with cgroups.
As for having more users than sockets, is that really likely, considering there are 64000 available ports?
The distinction between the long running processes and microflows is subtle but important. When started, long running processes wait for an input, execute some code when an input is received, and then continue waiting. In contrast, microflows are started once an input is received, and are terminated by the platform after the code in the microflow finishes executing. One way to describe microflows is to say that they are reactive, in the sense that they react to the incoming data.