"simply" is one of those words that makes people twitch... See, the packaging format is not the problem. The protocol is where it gets tricky.

I previously imagined Go support would arrive with a JSON on stdin/out/err protocol. Instead, your binary runs as a service with RPC endpoints. Unfortunately, it's a bit Go-oriented, so you'd have to implement some Go things (go/rpc, gobs) in your language/framework of choice to use it.

But, you get a (much) lower warm function startup cost because it's just an RPC call, all of which is neatly abstracted away for the Go programmer in the runtime library. Great for Go, bit of a bummer for those of us hoping for something more agnostic.

Yes, Lambda could support "Docker containers", but the more interesting problem to solve is the runtime model and protocol.

> bit of a bummer for those of us hoping for something more agnostic.

If you can imitate the go/rpc protocol, you an run any binary from any language on this, including LLVM based systems.

A version of this is already in play in https://github.com/apex/up

You are looking for a different product. What you are looking for is ECS+Fargate. That does exactly this. The difference is that with ECS+Fargate you are in charge of building the entire application from scratch. With lambda you just implement 1 function, the "service" part of your app and all the plumbing to support different things like X-Ray, logging, handling errors, hot and cold boots, etc etc are handler by AWS.

I think Lambda should do this. Lambda seems like its a more managed version of the concepts in inetd + CGI: Like inetd, Lambda will start a server, if needed, to handle a new connection, and like CGI, Lambda passes request data to user application code in a standardized way. You could implement a Lambda-like service using inetd + CGI to run functions in Docker containers, and then destroy the container once you get the result.

Spawning a process in a container is expensive. Not only do you need to wait for the container to init and mount its filesystems, but you then need to wait for the inner application to boot on every request, on top of that. So, allow containers to linger with applications booted, like Lambda does to save on those starts.

Since we aren't launching a new process for every call, we can't use CGI with its environment variables, etc, so we'll have to describe a new protocol for calling your functions, like... HTTP! The interface could be POST /call with JSON params.

I'm not sure how this model would differ semantically from the fully-managed, in-process system that Lambda uses. The management system can always kill containers that go over their memory limits or seem to be doing suspicious amounts of system calls when they're not executing a function. AWS can still go about providing in-process libraries for "supported" languages that implement these protocols, but if they also describe these protocols, then the community can go ahead and develop libraries for other languages that could eventually be adopted and officially supported.

In fact, the community is already producing libraries that use AWS's supported languages to allow hosting lambdas in other languages, via a CGI-like model! Just look at https://github.com/apex/apex - a NodeJS shim will spawn a command in the language of your choice, and then return the result back to Lambda.

Why would a container be slow to create? It's effectively a fork with namespacing options. The overlay file system is just a product of one or more layers that are already computed; there's not much complexity to its mounting compared to something like ext4 or NFS. I haven't timed it, but containers seem damn fast to start with Docker and Kubernetes.

> I haven't timed it, but containers seem damn fast to start with Docker and Kubernetes.

You should time it then, it's incredibly slow these days

    $ time docker run busybox /bin/true
    .388s total

    $ time docker run --net=none busybox /bin/true
    .310s total

The above times, of over 300ms, are with docker 17.06 and the overlay2 driver on a powerful computer with an ssd.

It's horrifically slow because the entire docker codebase is disgustingly bloated.

It has to fork something like 10 processes and synchronize between them (for everything from network initialization to runc hooks). It has to make ipc calls from docker-cli to dockerd to containerd and back again, allocating and freeing large chunks of memory for grpc along the way.

It's incredibly slow.

On the other hand, just forking and running something with namespacing options is effectively instant:

    $ dir=$(mktemp -d)
    $ docker run --name bbexport busybox /bin/true
    $ cd $dir
    $ docker export bbexport | tar x
    $ time nsenter --root=$dir /bin/true
    0.000s total

This is the reason they don't let you use arbitrary docker containers; docker containers are a ton more than a fork and some options. They're a bloated moving target.

What lambda does allows them to optmize much more. In fact, lambda doesn't even have to fork a process sometimes because they can keep the e.g. javascript runtime running and then just add a new request into the existing process directly. They can freeze and thaw environments from known good states for request processing.

But most importantly, by not being compatible in any way with docker, they don't have to deal with the bloated mess that is docker and the ecosystem around it.

They don't have to load 3 daemons measuring 50MB into memory to run 100kb of javascript (which they can currently run more quickly than docker can even start the simpleist container)

Even though a container starts "damn fast" in Docker, you're still adding significant latency to each call of your function if it requires a new container.

    $ time docker run alpine:3.6 sh -c exit
    docker run alpine:3.6 sh -c exit  0.02s user 0.02s system 3% cpu 1.191 total
    $ time sh -c exit
    sh -c exit  0.00s user 0.01s system 94% cpu 0.007 total

Then add to that the cost of loading your function's code into memory. For an interpreted language, that is probably going to take a while as the runtime does a bunch of system calls and parses text files. Even for a compiled, statically-linked language like Go, the executable (a random binary I picked was 50MB) needs to be copied into RAM.

For this reason, Lambda tries to keep your containers/function-wrapper-applications booted up in RAM: https://aws.amazon.com/blogs/compute/container-reuse-in-lamb...

True, but that's Docker. There's nothing, that I know, that would make Linux containers inherently much slower than a fork. Amazon could conceivably also write their own container runtime that kept the container ready to run without actually starting the app (though I'm sure there's a lot more to it -- I don't know how Lambda is implemented).

> I'm still wondering why Lambda simply doesn't support Docker containers? Would put an end to all this requests like "Please support $my_favorite_language on Lambda"

Because they can present that as a separate service (Fargate) and bill more for it.

Fargate isn't exactly what you're describing, but it's a close match. It's in Amazon's best interests to charge people based on the amount of "DIY" they're willing to put up with, and that's why they offer a number of overlapping services at different price points.

Amazon does not launch a brand new container for every invocation of lambda, only cold starts. Supporting arbitrary Docker containers is definitely more costly.

Because it’s a function execution system, not an application execution system. If you need the latter, you can use ECS or K8S.

Fargate is probably the closest AWS offering.

This, also, is the direction the industry is going, regardless of cloud provider.

Isn't that what Fargate is, basically?

Docker executes long running processes that communicate over HTTP/TCP. AWS has ECS, Fargate and has announced K8S support soon.

Lambda is for executing individual functions, which implies that the execution environment must be language aware (could get away with JVM dynamic invoke or the .NET equivalent as well maybe). The way parameters are passed in, the concurrency model etc are all language specific.

And yet, the golang runtime for Lambda is a long* running process that communicates over TCP. :-)

It would be entirely possible to create a language agnostic Lambda runtime model, be it stdin/out/err, or TCP.

* relatively so

That's already around https://github.com/apex/up

Not as a wrapper, but an official Lambda runtime model. This Go release points to a much more interesting way of doing so (RPC) than what I had previously imagined (stdin/out/err).

Yeah, the problem with RPC is that there's no one RPC to rule them all, like there is with JSON or HTTP. They'll have to build RPC wrappers for each language / VM, and choose the most popular RPC packager if there's isn't one in the stdlib.

Couldn't they pick an existing, established cross-language RPC system like Thrift or gRPC?

I don't know about the internals, but my guess is one of the following:

1) there's something about the lambda platform that just doesn't lend itself well to customer provided containers. a lot of people would AT LEAST have issues keeping image size reasonable.

2) they plan to add this. AWS is already investing in serverless container services, take a look at Fargate.

Like many people stated here, Fargate will give you this. Lambda is actually under the hood a container system.

It seems that they benefit from lock-in, whether it's deliberate or not.

There is no lock-in... you can write an agnostic function for both AWS lambda and azure function.

I'm aware of tools like Serverless. They help remove the lock-in in some situations. That doesn't mean there's no lock-in to speak of at all.

In regards to C# functions, there is no lock-in to AWS, it's a piece of code, you can run that function on AWS Lambda, or in ASP.NET Website, or Azure Function, or console app...

It's literally a method which has a single parameter class, and a single class returned...

Even if you're using node, java, golang, python. Other than the packaging. How is there any lock in?

Aren't you spending resources setting up your CI/CD pipeline /deployment tooling to talk to Amazon's APIs?

Unless all of the providers have the same API to deploy functions, they are all trying to lock you in.

So not lock-in at the code level (unless your serverless app talks to S3 or whatever Amazon services), but lock-in of your whole devops process.

So first it's all "Oh you're locked into IIS" and "You're locked into Windows", then "You're locked into AWS" now it's all "You're locked into your CI/CD pipeline"

It sounds like, no matter what you do, you're locked in. Using Docker? Ah you're locked into Docker. Wrote your app in PHP, on you're locked into PHP now.

I just don't understand this whole 'lock-in' mindset, it's just absurd now.

>I just don't understand this whole 'lock-in' mindset, it's just absurd now.

If you don't understand it, then it's probably not absurd.

Look into the whole free software movement and the history behind it (e.g. the evils of Microsoft in the 90s).

If a vendor can change a price on something that forces you to pay up or lose your system, you are locked in.

This isn't just some open source ideology without impacts either. It should be immediately obvious to anyone with any business acumen why making yourself beholden to the decisions of a single vendor is a bad position to be in.

The vital difference being that Docker and PHP aren't subscription-based, and aren't controlled by a single corporation.

Replace Docker and PHP with anything and the same is true tho.

If you write your application in any language, or use any framework or library, you're locked in, by today's standards.

The difference is that you can't be screwed by a single company deciding to change prices. When people refer to "lock-in", they are referring to being able to be screwed by a single vendor.

Nobody cares if they are locked into x86, tcp/ip, and http. If you don't understand this difference as an engineer, you end up causing a big risk to your company (or clients) without even knowing it.

Oh so now it’s paying for something that’s lock-in. This lock in sure keeps changing.

If you look at the amount of work Serverless has to do, packaging can generate quite an amount of lock-in. It is hardly any lock-in in the simple cases of grabbing a single function and deploying it where you want.

The architecture of lambda may prevent this.

> The architecture of lambda may prevent this.

It doesn't; you've always been able to use any language you want, by embedding it within the runtime of a supported language. This is just announcing proper, official support for it instead of having to rely on a workaround.

Lambda's execution model is language-specific runtimes being fed events off a queue through a single well-defined RPC or RPC-like entry point, with the process frozen or discarded upon queue exhaustion.

So the architecture of Lambda may indeed be a severe mismatch for running arbitrary containers.