At which point we're paying double just to work around cold start latency, which was the original point.
So if you're at 50% capacity with lots of idle time, it can scale to 0.
Scale to 0 doesn't mean "cold". One easy way to keep both Cloud Run and Container Apps "warm" but functionally scaled to 0 for costs is to just ping it every few minutes with Cloud Scheduler set up on a cron pointing to a health endpoint.
This diagram helps visualize: https://www.gstatic.com/bricks/image/5eb3c8e9-6159-4e2f-ba98... (gray bar at the top shows when the instance is started and when it ends; blue bars show actual request, and green bars show the billing)
If there's a 10s gap between requests, you are not billed for the 10s gap, but it is still effectively hot; instance does not stop. Request that comes back in 10s later does not have a startup penalty because the instance is still up.
How do you leverage this to build a high throughput system with no cold starts? Just prime it with a short single request (e.g. `/health`) before instance is stopped.
The container instances auto-scale by default to a configured n (0 - n) with the option to define specific scale points. If I'm sensitive to high bursts, this can be a matter of scaling at 50% capacity and a warm instance is available while the application still has headway.
Cases where this falls apart: DDoS in which case, I argue that 1) it is better in any case that it falls apart rather than absorbing that type of traffic spike, 2) such traffic is better filtered upstream in any case in an API Gateway + WAF.
I have worked with both types of solutions for different use cases (was invited presenter at Azure Serverless Conf[0]), but my take nowadays is that serverless containers are almost always a better approach since it provides much more flexibility.
[0] https://learn.microsoft.com/en-us/shows/azure-serverless-con...