Re:Logical replication and decoding for Cloud SQL for Postgr...
by gunnarmorling on Saturday June 05, 2021 @06:04PM
(#27408790)
Why the Parquet step? You should be able to do straight Debezium -> Kafka -> BQ, using the BQ sink connector for Kafka Connect (https://github.com/confluentinc/kafka-connect-bigquery); we have users using this with the Debezium MySQL connector, I'd expect this to work equally for Postgres.
Disclaimer: working on Debezium
Debezium => Kafka => Parquet in Google Cloud Storage => BigQuery external queries.