Re:Apache Kafka Goes 1.0
by threeseed on Wednesday November 01, 2017 @03:06PM
(#15604503)
Remember that both Kafka and Kinesis have "at least once" delivery and from experience it's a common scenario. You will receive multiple messages. Two options: (1) store hashes of each row and then deduplicate, (2) use a database.
I can't remember if it's an issue for change capture since in theory you have some insert timestamp column anyway.
I can't remember if it's an issue for change capture since in theory you have some insert timestamp column anyway.