News for hackers. Stuff that matters.

A comment on Twitter’s Kafka adoption story. Parent comment

Re:Twitter’s Kafka adoption story by majidazimi on Thursday November 29, 2018 @04:38PM (#18564298)
Both Pulsar and LogDevice solve the problem of far-away-consumers with kind of segmenting the log and storing each segment in separate machine to reduce disk seeks and page cache pollution (when c1 reads head of the log and c2 reads tail).

So technically they are better than Kafka. But as the article mentions, they just used SSDs to get around this issue in Kafka. By looking at article the only valid reason to switch to Kafka seems to be KStream.

[ Parent | on HN ]
Re:Twitter’s Kafka adoption story by manigandham on Thursday November 29, 2018 @05:04PM (#18564538)
Pulsar has a much better storage system that scales independently so latency stays low regardless of consumer offset. It can also now tier to S3/cloud storage natively and it has many other features like supporting millions of topics and per-message acknowledgements.
[ Parent | on HN ]