Re:DBLog: A Change-Data-Capture Framework
by rammy1234 on Saturday February 15, 2020 @03:28PM
(#22337755)
Added a kafka layer to make this real time capture. I guess lot of people are trying to do this. I guess what is the keypoint i am missing here.anyone ?
[ Parent
| on HN ]
Re:DBLog: A Change-Data-Capture Framework
by srijan4 on Saturday February 15, 2020 @03:49PM
(#22337884)
Kafka or similar is already used by the existing solutions mentioned in the post.
I think what this adds is a better way of dump processing without using database specific features.
Re:DBLog: A Change-Data-Capture Framework
by aandreakis on Sunday February 16, 2020 @12:49AM
(#22340011)
DBLog has a very simple Output interface which allows to plugin a writer into whatever Output is desired: a stream (like Kafka), a datastore, a service, ...
For example one can use MySQL as a source and have ElasticSearch as a direct output, without needing to go through an intermediate stream like Kafka.
The described properties of DBLog (see blog post) hold true regardless of the output, including capturing changes in real-time and writing them to a desired output.