After Raft, it became easier to just implement that layer yourself and so most projects after Raft (or probably more accurately once people started seeing how stable etcd was, ~2014), just used Raft internally where they would have previously used zookeeper.
IMHO, it's still easier to delegate the consensus problem to a third party service like Zookeeper or ETCD.
Also, do any of those projects publish their Raft implementation as a library for other projects to include?
Most NoSQL databases, now, use Raft, which didn't exist at the time when Kafka was created. Other NoSQL databases, at the time, were not as stable as Zookeeper or had silent bugs that ate data (see aphyr's Jepsen series[1], which thourghly tested several NoSQL databases and found many to be failing, except for Zookeeper).
> Yeah! I mean, I find a lot of linearizability errors in various databases, but this was also my very first time doing this kind of test, and it varies from system to system. Could have easily slipped through the cracks.
In summary, aphyr thought Zookeeper is linearizable even though it doesn't provide linearizable ops.
Looks like Zookeeper needs to be tested again.