What every software engineer should know about Apache Kafka

Re:What every software engineer should know about Apache Kafka by pacala on Saturday May 16, 2020 @09:13PM (#23208571)

Not super familiar with Kafka, but behind the scenes some RDBMS are log replay engines. Are there technical reasons to not slap a RDBMS frontend on top of a Kafka log and avoid dealing with raw streams in the application code?

Edit: From the sibling comments, the answer is https://ksqldb.io.

[ Parent | on HN ]

Re:What every software engineer should know about Apache Kafka by gen220 on Saturday May 16, 2020 @09:57PM (#23208818)

The technical reason is the inherently distributed and shared nature of Kafka setups makes writing an ACID-compliant frontend (basically?) impossible.

If your application assumes ACID compliance, it will fail. Also, I haven’t looked into this one as directly, but I imagine that query latency compared to an RDBMS would be quite poor.

IMO, most people reach for Kafka too soon, where an RDBMS with a proper schema that includes transaction and valid history would better suit the domain. It’s possible to write very efficient and ergonomic “append-only”, temporal data schemas in Postgres schemas.

[ Parent | on HN ]

Re:What every software engineer should know about Apache Kafka by hashhar on Sunday May 17, 2020 @12:08PM (#23212882)

But there are a lot of usecases where ACID is not needed and we can do with eventually consistent states. Btw, you do get C and D.

[ Parent | on HN ]