Re:Ask HN: How was your experience with AWS Lambda in produc...
by gallamine on Wednesday June 21, 2017 @08:35AM
(#14603159)
Can you clarify this point:
> its strength is in distributed processing; in other words, long-running CPU-bound apps.
it seems to me that's an explicit non-usecase for Lambda given it limits sessions to < 5min per invocation.
Re:Ask HN: How was your experience with AWS Lambda in produc...
by scrollaway on Wednesday June 21, 2017 @08:51AM
(#14603280)
I should have been more specific yes. I actually meant CPU-or-network-bound tasks with a predictable, sub-5min runtime.
Like I said we use it for game replay processing, so that's 5-15 second tasks that read and parse log files and hit a db and s3 with the results.
Other suitable tasks: image resizing, API chatter, bounded video transcoding, etc. Lambda is pretty good at distributed processing (as long as you can make the bill work in your favour, over hosting your own overprovisioned fleet).
Re:Ask HN: How was your experience with AWS Lambda in produc...
by weeefun on Wednesday June 21, 2017 @08:16AM
(#14603031)
Regarding the problem that you want to read the costs per function: have you considered using resources tags? This could help a bit when evaluating your costs (though not directly on the bill). https://aws.amazon.com/answers/account-management/aws-taggin...
Re:Ask HN: How was your experience with AWS Lambda in produc...
by scrollaway on Wednesday June 21, 2017 @08:52AM
(#14603288)
I've read about them yeah. A friend also showed me recently the power of stuffing the advanced billing CSVs into Redshift and querying them with Tableau or something.
Personally, I think it's all fucking ridiculous the amount of effort you have to spend into reading your own bill.
My #1 concern with it went away a while back when Amazon finally added support for Python 3 (3.6).
It behaved as advertised: Allowed us to scale without worrying about scaling. After a year of using it however I'm really not a big fan of the technology.
It's opaque. Pulling logs, crashes and metrics out of it is like pulling teeth. There's a lot of bells and whistles which are just missing. And the weirdest thing to me is how people keep using it to create "serverless websites" when that is really not its strength -- its strength is in distributed processing; in other words, long-running CPU-bound apps.
The dev experience is poor. We had to build our own system to deploy our builds to Lambda. Build our own canary/rollback system, etc. With Zappa it's better nowadays although for the longest time it didn't really support non-website-like Lambda apps.
It's expensive. You pay for invocations, you pay for running speed, and all of this is super hard to read on the bill (which function costs me the most and when? Gotta do your own advanced bill graphing for that). And if you want more CPU, you have to also increase memory; so right now our apps are paying for hundreds of MBs of memory we're not using just because it makes sense to pay for the extra CPU. (2x your CPU to 2x your speed is a net-neutral cost, if you're CPU-bound).
But the kicker in all this is that the entire system is proprietary and it's really hard to reproduce a test environment for it. The LambCI people have done it, but even so, it's a hell of a system to mock and has a pretty strong lock-in.
We're currently moving some S3-bound queue stuff into SQS and dropping Lambda at the same time could make sense.
I certainly recommend trying Lambda as a tech project, but I would not recommend going out of your way to use it just so you can be "serverless". Consider your use case carefully.