AWS Kinesis with Lambdas: Lessons Learned

Re:AWS Kinesis with Lambdas: Lessons Learned by teej on Friday July 13, 2018 @01:04PM (#17524690)

The lambdas are only triggered when there is data in the stream, so the "consumer" in this case is not always running. Under the hood there is a process that polls Kinesis to check for records but it's completely managed by AWS and you aren't charged for it.

Plus, in any real world scenario, if you're running 24x7 your load isn't evenly distributed throughout the day. Which means you're setting up autoscaling (w/ added time+complexity) or provisioning for peak, wiping out your cost savings.

In my experience if your data volumes are low enough like in the article, a Kinesis+Lambda setup is stress free and quick to implement. That makes it worth the cost over raw instances.

[ Parent | on HN ]

Re:AWS Kinesis with Lambdas: Lessons Learned by djhworld on Friday July 13, 2018 @03:40PM (#17526028)

It also depends on what your business problem is.

If it's just doing simple ETL then it's probably OK, but if you need to do aggregations on the data, you're going to have a bad time, or end up implementing some sort of ersatz map-reduce framework in lambda.

[ Parent | on HN ]