Re:Amazon Kinesis Firehose – Simple and Scalable Data Ingestion
by jhurliman on Thursday October 08, 2015 @02:01AM
(#10351157)
The only complication with this approach right now is that Lambda scripts can't communicate with anything inside a VPC (such as most Redshift instances). I imagine they will fix this issue in the future though.
But to me it wouldn't make much sense to use Kinesis Firehose unless the fees were cheaper than what it would cost to utilize AWS Lamdba for the same work. I mean it can't be all that many lines of nodejs code to pop events off a stream, flush them into a tempfile in batches, HTTP POST those batches to S3 with appropriate error handling/retry logic.
Admittedly I haven't crunched the math but just by eyeballing their pricing I suspect it may be more expensive than using your own lamdba. I wonder if Kinesis Firehose is implemented internally as an AWS Lambda, it wouldn't surprise me.
I suppose that eventually the pricing on this service will drop once someone open sources such a lamdba, especially since installing a lambda is so remarkably simple.