News for hackers. Stuff that matters.

A comment on Creating an advanced search engine with PostgreSQL. Parent comment

Re:Creating an advanced search engine with PostgreSQL by valzam on Wednesday July 12, 2023 @07:51PM (#36703279)
Or a bit more 'web scale's use wal_insert and the outbox pattern to transactionally replicate events to a messaging system (Kafka etc).
[ Parent | on HN ]
Re:Creating an advanced search engine with PostgreSQL by lyu07282 on Wednesday July 12, 2023 @10:37PM (#36704402)
you can also just use a CDC to import from Postgres into Kafka using Debezium and then on the other end Kafka Connect to write into Elasticsearch, no need to write any code at all with this setup ymmv ofc
[ Parent | on HN ]
Re:Creating an advanced search engine with PostgreSQL by paulryanrogers on Wednesday July 12, 2023 @11:16PM (#36704630)
Sounds like a lot of moving parts though, unless the team is already familiar with maintaining all those parts
[ Parent | on HN ]
Re:Creating an advanced search engine with PostgreSQL by lyu07282 on Wednesday July 12, 2023 @11:44PM (#36704775)
if you already use Kafka anyway its a valid solution, I wouldn't introduce Kafka for this obviously. Also Kafka Connect and Debezium are mature well maintained open source projects in case that wasn't clear enough.

https://debezium.io/

https://docs.confluent.io/platform/current/connect/index.htm...

[ Parent | on HN ]