News for hackers. Stuff that matters.

A comment on AWS Kinesis with Lambdas: Lessons Learned.

Re:AWS Kinesis with Lambdas: Lessons Learned by sheeshkebab on Friday July 13, 2018 @10:07PM (#17528376)
Copying a billion records screams for streaming? do you mean like a couple of hundred gigs that could be copied overnight (and maybe transformed with some script)?
[ Parent | on HN ]
Re:AWS Kinesis with Lambdas: Lessons Learned by nikhilkuria on Monday July 16, 2018 @04:55AM (#17539729)
Rather than doing a one-time copy of the data, we have applications continually writing to our on-premise MySQL. This data has to reactively reach our datastore on AWS. Also, this is not a 1-1 copy. There are several transformations and cross-validations in play.
[ Parent | on HN ]