thanks for your work on debezium! cdc to kafka has been a game changer for our data pipelines, so much easier to make generalized tooling at this level. our old pipeline requires either immutable rows with autoincrement primary keys or last modified dates with a secondary index on every table. it doesn't capture all the changes for example if a row is modified twice before the pipeline sees it the first update won't be recorded in the history. it also doesn't propagate deletes. debezium solves both of those issues plus doesn't require the application developers to do anything special for their data to get into the analytics systems. now instead of sounding like a broken record telling people to add last modified dates to all their tables i can sound like a broken record telling people to stop violating single source of truth by dual writing (which is a much better problem to have as before i had given up on single source of truth even being possible in a large organization)

That's awesome, very happy to hear! Drop by any time on the mailing list or chat if you got any issues or questions to discuss around Debezium.

This is a neat pattern, but it honestly just seems like you’re trading where you want the complexity. The outbox pattern would force devs to not use SQL directly for mutations - in so much as they can’t directly modify records like they normally would. I can see this being ok if you had someone with strong DBA skills implementing the abstraction.

Devs modify records just as they normally would. Only, in addition, they produce a record in the outbox table, representing the change to the outside world. I.e. instead of writing to the DB and Kafka, both changes are done to the database, and then events from the outbox table are CDC-ed to Kafka.