Kuvasz-streamer: open-source CDC for Postgres for low latency replication

Re:Kuvasz-streamer: open-source CDC for Postgres for low lat... by kuvasz-io on Tuesday January 07, 2025 @04:18PM (#42627619)

The date fields are actually timestamps having microsecond accuracy. Maybe this was not clear in the docs.

       1 | 12  |  1 | r1   |        | 1900-01-01 00:00:00+00        | 2025-01-07 21:15:49.233384+00 | f
       7 | 12  |  2 | r2   |        | 1900-01-01 00:00:00+00        | 2025-01-07 21:15:49.233384+00 | f
      13 | 12  |  1 | x1   |        | 2025-01-07 21:15:49.233384+00 | 9999-01-01 00:00:00+00        | f

[ Parent | on HN ]

Re:Kuvasz-streamer: open-source CDC for Postgres for low lat... by gregw2 on Thursday January 09, 2025 @04:34PM (#42650006)

Ah yeah, makes sense. The docs are misleading.

The tricky thing with DateTime SCD2s vs Date-only SCD2s is that DateTime SCD2s work for identifying what was true for a given click/transaction/ingest time, but doesn't work for identifying what was the final truth associated for a given "business date" such as an "invoice date". That tends to take an ETL or SQL window functions complexity/performance hit. But with streams/CDC, DateTime SCD2s are the easy+clean thing to implement.

Do you use the DateTime the message is received on the target system, or some DateTime from the originating journal/WAL-log?

[ Parent | on HN ]