News for hackers. Stuff that matters.
Re:KSQL: Open Source Streaming SQL for Apache Kafka by coolio222 on Monday August 28, 2017 @02:52PM (#15119058)
I encountered that problem. The ad hoc fix, was to have a version field in each event and functions that translate the old event into new event(s). The code that processes the events only processes events of the current version. If your old events had been denormalized this might result into repetition of events when splitted.
Re:KSQL: Open Source Streaming SQL for Apache Kafka by Joeri on Tuesday August 29, 2017 @03:18AM (#15122677)
To add to that, you can treat it like you would schema migration on databases: implement v1 to v2, v2 to v3, etc... and replay the migrations in order to migrate from whatever version of the event is to the latest version. This allows keeping event migration code as immutable as the event versions it migrates between.
Re:KSQL: Open Source Streaming SQL for Apache Kafka by daddykotex on Monday August 28, 2017 @02:54PM (#15119081)
Ok, thanks for the pointer