Kafka Removing Zookeeper Dependency

Re:Kafka Removing Zookeeper Dependency by skyde on Sunday May 17, 2020 @01:40AM (#23209846)

so this use raft for topic data not just metadata ?

[ Parent | on HN ]

Re:Kafka Removing Zookeeper Dependency by agallego on Sunday May 17, 2020 @02:02AM (#23209929)

correct. we use one protocol, just raft - both for metadata and for data.

Raft is really easy to parallelize and dispatch to multiple followers async. I measured recently on 3 i3.8xlarge instances which give you 1.2GB/s - and i got around 1.18GB/s sustained -https://twitter.com/emaxerrno/status/1260415381321084929

Also, what's nice about using raft is that if there is a bug, we know it's w/ our implementation and not w/ the protocol. so it gives users sound reasoning.

[ Parent | on HN ]

Re:Kafka Removing Zookeeper Dependency by skyde on Sunday May 17, 2020 @02:07AM (#23209944)

are you hiring :-) I would love to work on this.

I consider myself a RAFT expert and worked in kafka for the past 5 years.

[ Parent | on HN ]

Re:Kafka Removing Zookeeper Dependency by agallego on Sunday May 17, 2020 @02:15AM (#23209973)

[email protected] - let's chat. i'm looking for a senior member atm.

[ Parent | on HN ]