Re:How to stream PostgreSQL CDC to Kafka and use Propel to g...
by jabart on Tuesday April 02, 2024 @03:14PM
(#39909719)
We built a streaming CDC pipeline in MSSQL and have aggregated joins to a flat table on the analytics database side. It's like 80 lines a table with half of that being a bulk load from the beginning of time in case we ever need to add a new columns or change schemas. We built it as a command line argument though I'm sure others do something similar.
80 lines could be trimmed down if we made some base class abstraction but it shipped and we don't touch it and moved on to other features.
80 lines could be trimmed down if we made some base class abstraction but it shipped and we don't touch it and moved on to other features.