Agreed. Perhaps the first thing a "best practices" post should start off with is what kinds of problems serverless is best suited for.

I agree with this. Having a traditionally relational store can be a huge help: lots of people know it well, the disadvantages are well known, and everything is insanely mature. I dislike how the article tries to say that isn’t serveless’s fault when it is just a trade off.

Yeah, RDBMSs do have some big advantages; one of them being the plethora of query patterns possible.

If there are other parts of your application which are consuming the same data but in different patterns, it might not be necessary that systems like DynamoDB work in those cases. You might need to make several secondary indexes in those cases and all of them require provisioning of their own. The other alternative being setting up some ETL layer which transforms the data and sends to appropriate sinks, from where they can be consumed; but for small services that adds more moving parts and cost to the system.

> It’s a shortcoming. Own it.

Well, I don't know much about Oracle and the other players, but AWS at least tries to own it with Aurora Serverless.

Moving away from RDBMS just wasn't an option for us. But AWS serverless provided a couple benefits:

  1. Aurora Serverless RDBMS (In pilot)
  2. Lambda function property caching (for which we cache connections)

How does Aurora Serverless help with the issue of having to reconnect for every lambda? (I am unfamiliar with the Aurora Serverless offering)

Also, can you elaborate/ point me to info on 2?

Really quickly then -

Background info: A lambda that spins up stays live for a while, and can continue to handle requests. The function handler is called again, so anything scoped to it, will disappear with the function, but anything scoped outside of the function will persist.

Ergo, move your connection creation outside of your handler.

Realistically you may need some abstraction to properly close and reopen the connection in the event of failure (so you can't have a 'bad' lambda, where something has happened to the connection and you can't fix it without deploying code again, to trigger a lambda refresh), but for the most part it just works.

Keeping connections open on a lambda isn’t a good thing. If you have a lot of Lambdas and a lot of load you can end up hitting max connection limits. Better to open late and close early and take a hit rather than risk taking your db offline.

Oh, agreed; it doesn't bottleneck on the connection but instead keeps creating them which can take performance down when you spike. Hence why the original article speaks of using something other than a RDBMS, such as Dynamo, since there really is no way of addressing that issue (since you can't set a concurrency limit on one kind of function; else you could have one function that has high concurrency invoke one that is far lower, that exists just to keep connections pooled).

If you absolutely must hit an RDBMS, be quick about it to free up the connection, and definitely consider if the latency matters that much; a few dozen ms more to initialize and close the connection is going to be a LOT cleaner.

But that said, how badly it penalizes you for scaling connections up depends on the DB.

Interesting. I'd considered this, but was worried about failure cases. Seems tricky.

You don’t need a reconnect for every invocation actually, only per instance. I know you don’t want to think about instances with lambda, but actually they just run on some VM and you can just open one connection per VM.

We use lambda functions(node with pg) to process incoming data. We just connect them to Aurora(Postgres) and make the inserts. We have never had any problem with the process whether we are pulling from a queue or other data source.

So why is rdbms and serverless seemingly mutually exclusive? Are rdbms queries really that slow compared to something like dynamodb?

Actually RDBMS queries are quite fast. But those systems weren't designed for use patterns which serverless applications typically have. Like MySQL will limit you on connection count and each new connection puts extra overhead on the server. So if every function negotiates its own connection you might eat away DB resources just on negotiating connections even though the DB queries were fast.

A couple of concepts core to the serverless model is that components communicate via API, and that resources are consumed on demand. Traditional RDBMS don’t really fit into that model. Google Spanner is the only thing I’ve seen that seems to come close. AWS doesn’t have much aside from Dynamo, especially considering how poorly lambdas perform when you put them inside a VPC.

As far as I know, most RDBMS are connection based, which brings an overhead. When you got your connection they're quite fast.

I wonder why all the predominant RDBMSes are heavyweight connection based, and why this is presumably so hard to change. There's nothing in SQL that requires this to be the case, at least conceptually.

TCP/IP performs a handshake to establish connections so there will always be some per-connection overhead required for this, no matter the service on the other end.

Beyond that, MySQL for example pre-allocates a fixed amount of memory for read, join and sort buffers per connection to efficiently handle different types of queries, though the buffer sizes are all tunable to optimize performance. There is also some OS-scheduler overhead involved in creating and running a separate thread per connection. A thread-cache optimizes the case where lots of short-lived connections are being created. (I dont have direct experience with PostgreSQL but I read that it forks a separate process per connection, which would presumably incur a significantly higher overhead than threads.)

Beyond this, any more specific discussion requires quantifying connection 'weight' to dispel/confirm your superstition that 'all the predominant RDBMSes are heavyweight'. In an apples-to-apples benchmark, it's quite possible that the connection-weight of some RDBMSes might actually be lighter than other database engines. I dont know myself, but would be interested if anyone has any such data to share on this.

It seems like pretty much everything was connection-based a few decades ago, when those systems were written. This can lead to some behavior in legacy software that causes big problems in cloud environments, or really any environments where connections are transitory.

The problem really boils down to the tight coupling between the HTTP layer and the rest of the stuff all the way down the stack. If you assume connections are always long-lived and stateful, you can make performance and memory optimizations by reserving memory buffers and threads for the sole use of a single connection, and using threadlocals for storing session data.

But if each connection gets its own thread (or thread pool), idle connections lock up resources and can prevent new connections from being opened, because all of the threads are currently allocated to existing connections. And setting up every new connection is a bit expensive and wasteful if it's only going to be used for one request.

I think the industry trend toward transitory connections, and even transitory application instances a la Kubernetes pods, is a good thing for a lot of reasons. If nothing else, the knowledge that connections can be short-lived leads to more fault-tolerant software because the business logic layer cannot depend on assumptions about the HTTP layer. The big downside is that a lot of older stuff isn't really suitable for that kind of world, serverless or not, and it can be really hard to refactor.

Part of it is legacy but also, interactions with an RDBMS are often stateful so it's useful to have a session (in which such things as transactions can live, etc). I'd guess there isn't much reason to change this because there are standard and effective workarounds that just haven't yet made their way to things like lambda.

Can you elaborate on what “standard and effective workarounds” you are referring to here?

Connection pooling is the most obvious one which is so common it tends to be transparent and built into many DB access libraries. Then there are all sorts of proxies/load balancers/multiplexors. If you think about it, the bits of code that handle web requests in your typical web app/framework/whatnot are very much like lambdas and when you write those, you generally don't have to worry about the cost of DB connections because it's a well-solved problem.

Saves overhead.

Crappy application-servers pump queries like nobodies business, so you need to shave every byte of them you can.

I don't get the difference particularly... You have to open a tcp connection to talk to dynamodb, right?

From the docs: DynamoDB is a web service, and interactions with it are stateless. Applications do not need to maintain persistent network connections. Instead, interaction with DynamoDB occurs using HTTP(S) requests and responses.

One could, if they wanted, open a new MySQL connection for every statement, which seems similar. One could do this asynchronously in the background too, simulating the eventually consistent nature as well.

I think the problem with that is the performance would be bad, because the database is not designed to be used thay way and those assumptions go as deep as the way it allocates memory.

True.

Maybe DynamoDB works better with this model, because it is build up without the idea of maintaining connections?

There’s many frontend servers to Dynamodb. Effectively the http endpoints are a connection pool to the actual backing datastore

To solve connect to a connection pool service instead of a RDBMS directly. E.g. Pgbouncer for Postgres

I think that a big part of Aurora Serverless being different from regular RDS, is something like this has been implemented in front of RDS. Havent used Aurora Serverless yet, but I think you can only talk to it by HTTPS, right?