News for hackers. Stuff that matters.
Re:AWS Lambda Cold Start Times by gunnarmorling on Tuesday October 12, 2021 @11:40AM (#28841096)
The best cold starts are those which aren't noticed by the user. For my blog search (which runs on Lambda), I found a nice way of achieving that [1]: as soon as a user puts the focus to the input field for the search text, this will already submit a "ping" request to Lambda. Then, when they submit the actual query itself, they will hit the already running Lambda most of the times.

And, as others said, assigning more RAM to your Lambda than it actually may need itself, will also help with cold start times, as this increases the assigned CPU shares, too.

[1] https://www.morling.dev/blog/how-i-built-a-serverless-search...

Re:AWS Lambda Cold Start Times by websap on Tuesday October 12, 2021 @01:40PM (#28842617)
Have you tried provision concurrency[1], this along with ARM based functions can help you with performance and reduce costs.

1 - https://aws.amazon.com/blogs/aws/new-provisioned-concurrency...

Re:AWS Lambda Cold Start Times by gunnarmorling on Tuesday October 12, 2021 @02:48PM (#28843506)
PC doesn't make that much sense for a use case like mine, where absolute cost minimization is the goal. My request volume (including ping requests) easily fits into the Lambda free tier, whereas with PC I'd be looking at some cost.

More generally, I feel PC kinda defeats the purpose of Lambda to begin with. If I'm in that range of continuous load, I'd rather look at more traditional means of provisioning and hosting this application.

Re:AWS Lambda Cold Start Times by zxcvbn4038 on Tuesday October 12, 2021 @04:02PM (#28844378)
I think you are thinking of reserved concurrency - with reserved concurrency you can only run as many instances as you have reserved. With provisioned concurrency you can run as many as your quota allows but you are guaranteed to be able to handle as many concurrent instances have provisioned. Neither one of these are going to help you with cold start times, it is not defining the number of instances running, its just reserving a portion of quota so that they can run. Where provisioned/reserved concurrency comes in useful is keeping a run-away lambda function from starving out other functions either by guaranteeing a portion of the quota for the other functions or keeping a function from exceeding a number of concurrent executions.

The "ping" technique you mentioned is one way to keep a function warm but if lambda decides to start a second instance of the function because the hot one is handling a request, then that person is going to take a warm up hit and nothing you can do about that.

If you are really latency sensitive then lambda might not be the right choice for you. You can get a t4g.nano SPOT instance for about $3.50/month and you can keep that warm, but that is probably a whole lot more then you are paying for lambda.

Re:AWS Lambda Cold Start Times by gjhr on Tuesday October 12, 2021 @04:26PM (#28844661)
Provisioned concurreny does decrease cold starts.

Reserved concurrency both guarantees a portion of your concurrency limit be allocated to a lambda as well as capping concurrency of that lambda to that portion. Reserved concurrency has no cost associated.

Provisioned concurrency keeps a certain number of execution environments warm for your use. Provisioned concurrency costs money.

Re:AWS Lambda Cold Start Times by itisit on Tuesday October 12, 2021 @04:36PM (#28844752)
Re:AWS Lambda Cold Start Times by websap on Tuesday October 12, 2021 @03:36PM (#28844068)
PC is a good way to keep certain amount of performance guarantees. It seems like you’re solution fits into the free tier, so probably a bad idea for you.
Re:AWS Lambda Cold Start Times by 015a on Tuesday October 12, 2021 @02:42PM (#28843433)
Provisioned concurrency is a bit of a non-starter for cold start latency reduction in user-facing applications. Its a tool in the toolbox, but the problem is: Let's say you set the provisioned concurrency at 10, then you have 11 concurrent requests come in. That 11th request will still get a cold start. PC doesn't scale automatically.

The ping architecture of warming up functions does scale (better) in this setup. Its not perfect, but nothing on lambda is; its possible that the ping request for User2 could get routed to a live function, which finishes, then starts handling a real request for User1, User2's real request comes in but this function is busy, so Lambda cold starts another one.

That being said, these ping functions can be cheap relative to PC. PC is not cheap; its ~$10/mo/gb/function.

With the ping architecture; you'd generally just invoke then immediately exit, so there's very little billed execution time. For sparsely executed functions, the ping architecture is better because PC is billed 24/7/365, whereas ping is billed per invoke (that being said: PC is a much cleaner solution). For high volume functions, PC may be cheaper, but it also won't work as well; if your concurrent executions are pretty stable, then its a great choice, but if its spiky high volume (as most setups are!) then it's hard to find a number which balances both cost and efficacy.

A setup which also includes Application Autoscaling, to automatically adjust the PC level with time of day or system load or whatever, would approach a better setup. But, of course, be more work.

The other frustrating thing about PC is its price relative to a cloudwatch events cron warming setup. Most sources say Lambdas stay warm for ~5 minutes, which means you only need ~15,000 invocations/month per "faux-Provisioned Concurrency" level. Theorycrafting at 100ms/invocation and 512mb: that's ~$0.012/month. PC is ~500 times more expensive than this. This kind of cron-based ping setup is very easy when PC=1; at 1+, it gets harder, but not impossible: You have a cron configured to execute a "proxy" lambda, which then concurrently invokes the real lambda N times depending on what level of PC you more-or-less want.

It's also worth noting that PC cannot be maintained on the $LATEST function version. It can only be maintained on an explicit Function Version or Alias (which does not point to $LATEST). So there's just more management overhead there; you can't just have a CI deploy functions and let AWS do everything else, you have to configure the CI to publish new versions, delete the existing PC config, and create a new PC config, maybe also adjust autoscaling settings as well if you configured it. All automatable, but definitely in the domain of "CloudOps Engineer Job Security"; anyone who tells you AWS will make your life easier is probably working for AWS.

Re:AWS Lambda Cold Start Times by websap on Tuesday October 12, 2021 @03:34PM (#28844046)
At least in the OP’s case it seems the requests are infrequent. I understand your point about the 11th request, but at that point what % of your customers are impacted. The tail latency is always an issue, PC is a good way to solve that.
Re:AWS Lambda Cold Start Times by 015a on Tuesday October 12, 2021 @04:07PM (#28844439)
Yeah, I'd say it just depends on your needs.

PC is definitely "better". It does require you to opt-in to function versioning, but if your ops automation is good enough to make that easy, its just better. The biggest advantage over warming pings is, really, that it keeps your metrics and logs clean and focused just on user invocations, not on weird faux-invokes that are extremely difficult to filter out.

But, its downside is that its ~100x more expensive than warming pings for infrequently invoked functions (defined as: average concurrent invocations over a month timeframe between 0 and 1).

If your company has the money, go for PC. But don't feel bad about going for warming pings; it ain't a bad setup at all.

Re:AWS Lambda Cold Start Times by WatchDog on Tuesday October 12, 2021 @06:17PM (#28845737)
Hit the nail on the head with this one. The pinging approach incredibly more cost-effective than provisioned concurrency.
Re:AWS Lambda Cold Start Times by buzzdenver on Tuesday October 12, 2021 @12:52PM (#28841998)
So effectively you're going to pay double?
Re:AWS Lambda Cold Start Times by musingsole on Tuesday October 12, 2021 @12:57PM (#28842073)
You increase the usage of the lambda, sure, but not by double unless all this lambda does is respond to the search. There's a point where this is all semantics when running cloud services with regard to how you've deployed your code behind gateways and lambdas, but most applications have obvious triggers that could warm up a lambda at 10% or less overhead.

And even then....lambdas are dirt cheap.

Re:AWS Lambda Cold Start Times by wpietri on Tuesday October 12, 2021 @02:00PM (#28842907)
Lambdas can be dirt cheap if you use them well. But that's always what happens. I recently saw a report about contractor who used them for crawling websites. Their client saw a surprise $12k bill for Lambda, when a simple Scrapy crawler on a low-end instance would have cost them < $100/month for the same load.

Why? Because if you spin up a ton of Lambda invocations and have them sit around waiting for the network, you pay for each CPU to sit idle, rather than having just one CPU stay busy managing a bunch of async IO.

Re:AWS Lambda Cold Start Times by rmbyrro on Tuesday October 12, 2021 @07:35PM (#28846426)
This is poor architecture, not Lambda's fault.

Same result would be achieved by paying tons of $$ for a high-end machine with dozens of cores, just to scrape one URL per core.

They could have implemented asyncIO within just a few Lambdas - heck even one Lambda could certainly handle hundreds, if not thousands of ASYNC jobs at once, just as a cheap machine, as you pointed.

They treated Lambda functions as threads or async jobs, while they should be looked at as core processors.

Re:AWS Lambda Cold Start Times by oblio on Thursday October 14, 2021 @02:52AM (#28861270)
I'm not sure I understand the point about asyncIO: yeah, the Lambda CPU isn't spinning, but don't they count actual runtime? So even if things are async, the Lambda still needs to be up to respond to the event when the async IO is finished.

Or do you mean handling things in an async way and Lambda passing the info on to another computing environment?

Re:AWS Lambda Cold Start Times by wpietri on Thursday October 14, 2021 @09:57AM (#28864253)
I think what they're talking about is using one Lambda to persist, handling many crawl requests. That doesn't make a ton of sense, as each Lambda has to make sure to push all its results and shut down before the max timeout is up, but it would technically work and be more efficient than what I described. Much better, though, just to use long-running spot instances, though, or just a regular instance running something like Scrapy.
Re:AWS Lambda Cold Start Times by wpietri on Tuesday October 12, 2021 @07:38PM (#28846450)
I didn't say it was Lambda's fault. Indeed, I said Lambdas can be cheap if you use them well. The implication being that some people don't.
Re:AWS Lambda Cold Start Times by rmbyrro on Tuesday October 12, 2021 @09:14PM (#28847117)
Sure. But isn't this valid to any tech? As I pointed, one could make stupid use of expensive machine with dozens of cores as well...
Re:AWS Lambda Cold Start Times by randomdude402 on Tuesday October 12, 2021 @10:01PM (#28847403)
Right, but at least with the expensive machine, one is making a conscious decision to spend a certain, known amount.
Re:AWS Lambda Cold Start Times by wpietri on Wednesday October 13, 2021 @12:35AM (#28848284)
Exactly. These people thought they were using the tech well. They weren't. I like stories of tech gone wrong as examples to learn from. I thought others might as well.
Re:AWS Lambda Cold Start Times by laurencerowe on Tuesday October 12, 2021 @05:16PM (#28845189)
Couldn't this also have been handled efficiently in Lambda had it been configured to use batching? (Assuming it was implemented as async invocations driven by a queue of some sort.)

It does seem unfortunate that Lambda cannot be configured to support concurrent synchronous invocations for serving HTTP requests concurrently.

Re:AWS Lambda Cold Start Times by dralley on Tuesday October 12, 2021 @08:31PM (#28846817)
This is a bit like asking if you can make a coal-rolling pickup truck more environmentally friendly by letting your friends carpool with you.

Technically, yes, but it's kinda missing the point.

Re:AWS Lambda Cold Start Times by laurencerowe on Tuesday October 12, 2021 @11:06PM (#28847804)
I guess you're paying a 5x premium for Lambda in the constant usage scenario, a similar premium as on-demand vs spot instances.

4GB m6g.medium $0.0385/hour

4GB Lambda works out at $0.1920/hour

It's been a while since I last used queue driven autoscaling groups, but https://aws.amazon.com/blogs/compute/scaling-your-applicatio... has startup time of about 4 minutes from scratch or 36s from a paused instance in a 'warm pool.' vs less than a second for Lambda in the original article. So the decision ends up coming down to responsiveness vs cost. Presumably Fargate comes somewhere in between.

Re:AWS Lambda Cold Start Times by aahortwwy on Tuesday October 12, 2021 @06:50PM (#28846088)
> assigning more RAM to your Lambda than it actually may need [...] increases the assigned CPU shares

"serverless" indeed

Re:AWS Lambda Cold Start Times by yoava on Tuesday October 12, 2021 @02:18PM (#28843141)
Just use http functions in a Velo by Wix project - you will get 50ms cold start out of the box.
Re:AWS Lambda Cold Start Times by gunnarmorling on Tuesday October 12, 2021 @02:52PM (#28843567)
Thanks, will take a look. Can I run Java, or any kind of binary, there? That's why I'm not running this with CloudFlare workers, for instance. An alternative I'm actually considering is fly.io.
Re:AWS Lambda Cold Start Times by yoava on Tuesday October 12, 2021 @04:01PM (#28844371)
Velo runs only JavaScript, but unlike cloudflare, velo runs a full node.js.

With that in mind, you can a lot of things, from accessing your database or any other asset on aws (or gcp or azure). You can also use other wix apis, like payment api or the wix data database.

Re:AWS Lambda Cold Start Times by rmbyrro on Tuesday October 12, 2021 @07:38PM (#28846448)
If you need to access a DB in AWS, isn't the networking overhead on every single call going to dominate all cold start performance gains in a few occasional requests in Velo?