News for hackers. Stuff that matters.
Re:Pulsar – an open-source distributed pub-sub messaging pla... by imglorp on Monday July 13, 2020 @06:37AM (#23819253)
Hasn't Kafka been talking about eliminating external zk forever? Are they making it an internal library or adopting a new consensus scheme?
Re:Pulsar – an open-source distributed pub-sub messaging pla... by miguno on Monday July 13, 2020 @07:07AM (#23819427)
Yes, you remember correctly. The Kafka community has already started the work on getting rid of ZooKeeper. See https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A... for details.
Re:Pulsar – an open-source distributed pub-sub messaging pla... by FridgeSeal on Monday July 13, 2020 @07:14AM (#23819486)
From my understanding, they’re going to use the brokers as their own metadata store, so the data that was in ZK goes in a topic instead, and I believe Raft for leadership?

KIP-500 is the proposal/PR (?) for ZK-free Kafka which I understand has been merged in and work is now progressing on follow up tasks.