Distributed consensus reloaded: ZooKeeper and replication in Kafka

Re:Distributed consensus reloaded: ZooKeeper and replication... by nano_o on Thursday August 27, 2015 @04:53PM (#10132016)

It seems similar to the primary-backup instance of the Vertical Paxos family. In the primary-backup Vertical Paxos, one can tolerate f faults with f+1 replicas as long as a reliable external reconfiguration master is there to replace failed replicas and make sure everyone agrees on the configuration. Here the external reconfiguration master would be ZooKeeper and the primary-backup protocol the ISR protocol.

http://research.microsoft.com/en-us/um/people/lamport/pubs/v...

[ Parent | on HN ]

Re:Distributed consensus reloaded: ZooKeeper and replication... by mjb on Thursday August 27, 2015 @06:22PM (#10132537)

Yeah, there are lots of similarities to Vertical Paxos. It also shares a lot van Renesse and Schneider's classic Chain Replication paper (https://www.usenix.org/legacy/event/osdi04/tech/full_papers/...), with the use of a configuration master to configure a chain that can't solve uniform consensus by itself.

That's not a criticism at all, though. There are a lot of good design and systems ideas there, and they are well explained in this post.

[ Parent | on HN ]

Re:Distributed consensus reloaded: ZooKeeper and replication... by fpj on Thursday August 27, 2015 @07:22PM (#10132768)

I'm glad you enjoyed the post. The chain replication work indeed relies on a paxos-based master service for reconfiguration, but I believe that's the only similarity, the replication scheme is different otherwise.

[ Parent | on HN ]

Re:Distributed consensus reloaded: ZooKeeper and replication... by fpj on Thursday August 27, 2015 @07:18PM (#10132753)

Thanks for the comment, the vertical paxos reference is indeed relevant here.

[ Parent | on HN ]