> You can generally resolve it yourself by poking seldom used functions to keep them hot.

We've tried this and it helps somewhat but when AWS attempts to scale your function based on load, cold starts re-appear. We've moved away from Lambdas where a dependable response time is required.

Can you explain more about why this is?

If you are experiencing cold starts it means that function is not used very often. If it's not used very often that likely means it's not user facing (or something less important like a Terms of Service page). If that's the case, why do you need instant response times?

No, that's not what it means. If you have high concurrent execution, you get 'cold start' every time the underlying service 'scales out' to support more. The MORE you use lambda concurrently, the more you hit the cold start issue. Granted, it's just for that one cold start execution per-scale node (and they could probably just optionally pre-fire to warm things in that instance, like with a cache), but it's definitely there horizontally.

I really with they would add an init() callback that is called on cold start but before any traffic is sent to your lambda. It wouldn't help when there are no lambdas running but it could be useful when things are scaling up, especially if you can ask for additional concurrency above the actual concurrency necessary for spikes.

More lifecycle events please! I'd love a onThawed and onFrozen or something so I can kill the DB connections neatly

You can already do this, I think. Just put this logic in your application's static initialization.

I don't think so. When it spins it up, the request is already in flight. Otherwise this would have been solved by everyone but instead everyone sees terrible cold start times.

Also, containers are restarted periodically (~4 hours) so even if you have very steady traffic you'll see cold starts regularly.

This is along the lines of what the other responses to this comment have said, but https://hackernoon.com/im-afraid-you-re-thinking-about-aws-l... gives a very detailed overview. It's titled "I'm afraid you’re thinking about AWS Lambda cold starts all wrong", because the way you're thinking about cold start times is common (and wrong).

that’s not entirely true. while your warm lambdas can and will take the traffic it your traffic ramps up, additional lambda instances will be spun up. you will pay cold start prices as they are spinning up. so, even if you have a heavily used lambda fn, depending on the traffic your p99 will still look pretty bad and you will not be able to guarantee that all requests will be processes in x ms or less.

https://github.com/awslabs/aws-lambda-cpp makes claims of millisecond cold start.

That's not for in-VPC functions, although if the underlying instance changes with firecracker migration users might see ENI start improvements. Currently your ENI usage is roughly floor(3GB/Memory Allocation)*(Concurrent Executions). If the 3GB changes users will see huge gains as each ENI creation can take around 9s.

Why would you use VPC functions?

Mostly due to being forced. For example: serverless aurora forces a VPC on you.

With delays as bad as 8+ seconds for ENI attachment, this is a significant holdback (https://medium.freecodecamp.org/lambda-vpc-cold-starts-a-lat...)

I believe you can now use Aurora Serverless outside of a VPC by using the new Data API endpoint.

You indeed can, but it's beta (us-east-1 only at the moment) and leaves a lot to be desired (https://www.jeremydaly.com/aurora-serverless-data-api-a-firs...)

Ah okay, I never thought about using an SQL DB.

True. In-VPC has extra costs.

I'm wondering how that's even possible if it includes the time for downloading your code from S3. I.e. normal cold starts (as I understand it) involve fetching the code from S3 to install on a VM. Perhaps they aren't including that time when claiming single milli cold start times?