You will have to analyze your burst rate and the lifetime of each request to figure out the size of your connection pool.
I also am not clear how AWS introduced the problem of not being able to connect to the DB. We've known about the need for connection pools and connection reuse from before AWS was a thing, no?
A process is given a connection from a pool when it requests the connection and holds it until it relinquishes the connection regardless of whether the process was in the logic or data access portions of their lifecycle. It is up to you as the developer to ask for the connection and to return it on an as-needed basis...Which is exactly what you should be doing in a lambda as well.
Just because a lambda container is persistent does not mean the connection given to it is stuck to that lambda even after that lambda returns the connection.
With a connection pool, you don’t need to hold onto a connection longer than necessary, but you are right that you could implement it in such a way that you hold it for life of the request.
Now if you somehow get a spike of requests to some endpoint that goes over your DB connection limit, suddenly all your newly scaled functions fail because they cannot get a connection to the database.
AWS added a service for RDS to deal with this, but it all just feels like a big kludge to deal with a problem that shouldn’t have to exist in the first place.