News for hackers. Stuff that matters.

A comment on Kafka Is Dead, Long Live Kafka. Parent comment

Re:Kafka Is Dead, Long Live Kafka by dottedmag on Monday August 07, 2023 @12:25PM (#37036621)
Do you have a reference documentation for S3 data layout?
[ Parent | on HN ]
Re:Kafka Is Dead, Long Live Kafka by richieartoul on Monday August 07, 2023 @12:38PM (#37036864)
(WarpStream founder here) Not currently. One of the things we're looking to do next is make it so any topic can be "automatically" turned into a standard format in S3, something like Parquet/Iceberg/Deltalake so its easier to consume for application that don't particularly care about the Kafka protocol.
[ Parent | on HN ]
Re:Kafka Is Dead, Long Live Kafka by EamonnMR on Monday August 07, 2023 @01:10PM (#37037381)
That would be a killer feature. Once we switched to MSK back at a previous gig, our biggest care and feeding task was adding new event types to our kafka-to-redshift ETL thing. Well, that and dealing with scaling our http wrapper...
[ Parent | on HN ]
Re:Kafka Is Dead, Long Live Kafka by bothra90 on Monday August 07, 2023 @01:28PM (#37037639)
That would be awesome!
[ Parent | on HN ]