ezrahoch

It is not an open-source project, but rather a proprietary implementation.

1) Keeping a log mechanism per bucket would be expensive, since it is a heavy mechanism. It will cost memory & network traffic. The additional information needed by Bizur for a bucket is minimal, basically, just a version.

2) In Bizur, rejoining the cluster includes the equivalent of copying the snapshot. However, that node can start participating in the Bizur even before it finished reading the entire snapshot, because of the key-value data model. Once a bucket has been copied, it can start serving it, which is much faster than log-based algorithms.

3) You are correct about the size of a bucket. In our implementation we aim to have up to 8-10 keys in a bucket, to avoid the overhead you mentioned. That's why it's especially important to be very efficient pet-bucket (see point #1).

Indeed, it is a tradeoff, and our solution doesn't require consistency across multiple keys.

More generally, any solution that doesn't require transactions across multiple keys, can use Bizur instead of classical Paxos, and achieve the same benefits (faster recovery from failures, higher concurrency, etc.)

Very nice work! Indeed we've done similar stuff :) I haven't read it all yet, I'll comment back once I do.

One of the differences between a raft group and a bizur bucket is the granularity and the overhead of that granularity.

Bizur buckets can be very small (10s of kb), and the overhead for them is minimal (10s of bytes).

What would be the overhead of having a raft group for every 32KB? Would it be practical / a good use of the machine's resources?

Single decree is a write-once algorithm. You can't use the same instance again, for another update of the associated value.

So just replacing bizur' should "register" implementation with a single decree paxos won't be sufficient. You'll need to do something more elaborate (creating more instances as you go along).

This kind of instance management isn't required in bizur.

I'm not sure what's the overhead off all that, w.r.t. memory foot print. Since this overhead is paid for each bucket, it might accumulate to quite a bit.