Re:Bottled Water: Stream real-time PostgreSQL change events ...
by anarazel on Thursday April 23, 2015 @12:44PM
(#9427994)
Cool. Since logical decoding got in I'd hoped somebody would write something roughly like this... I also didn't know about decoderbufs.
> Fortunately, you don’t have to install the plugin on your leader database — you can just use a follower (replica, hot standby) for that purpose. That way you can be sure that the plugin cannot corrupt your data, crash the main database instance or affect its performance.
That bit unfortunately isn't true yet. While we've worked hard that it'd be relatively easily possible to add that capability, it's not yet.
I do wonder why you chose to have a separate client that pushes the data to kafka, instead of doing that in a background worker. That way it'd integrate into the database's lifetime. I.e. it'd automatically get stopped/started when the database stops/starts, it'd get automatically (if you want) restart if it crashes etc.
> Fortunately, you don’t have to install the plugin on your leader database — you can just use a follower (replica, hot standby) for that purpose. That way you can be sure that the plugin cannot corrupt your data, crash the main database instance or affect its performance.
That bit unfortunately isn't true yet. While we've worked hard that it'd be relatively easily possible to add that capability, it's not yet.
I do wonder why you chose to have a separate client that pushes the data to kafka, instead of doing that in a background worker. That way it'd integrate into the database's lifetime. I.e. it'd automatically get stopped/started when the database stops/starts, it'd get automatically (if you want) restart if it crashes etc.