This looks interesting, looking forward to reading up on it. Would be really cool if it worked with SQLite ;)
Any chance JDK11 support will come for the Debezium Java core [1]?
Re:Cleaning Up Your Postgres Database
by gunnarmorling on Saturday March 06, 2021 @03:10PM
(#26370148)
Funny you should ask, we were just talking about raising the minimum baseline to JDK 11 the other day (currently it still is compatible with Java 1.8). That said, you definitely can use Debezium with Java 11 already; so perhaps you an entail a bit what exactly you mean by JDK 11 support here?
Re SQLite, is there any CDC interface which could be used to implement a connector?
Got you, that sounds great!
When the README listed Java 1.8 as a requirement, I assumed (perhaps incorrectly) an implication that it works with _only_ Java 1.8.
At my work, some critical libraries only work with Java 8 or 11, so the transition from 8 was challenging on the dependency versioning front. It's a gigantic project and codebase, though.
Re: SQLite, not that I know of, but I don't have deep knowledge on the matter. HN may go crazy (in a good way!) if there were a way to pull this off though!!!
Your log table will always be empty (so not taking up disk space), but the CDC tool will be able to get all the information from the TX logs (which themselves can be discarded after all replication slots have advanced a given offset). The advantage is that your log entries are transactionally written with the actual application data itself.
We use this technique for instance in the Debezium Quarkus extension for implementing the outbox pattern [1]. Postgres even allows you to insert records directly to the TX log via pg_logical_emit_message() - so no table involved at all -, but we don't support this in Debezium yet.
Disclaimer: I work on Debezium.
[1] https://debezium.io/documentation/reference/integrations/out...