News for hackers. Stuff that matters.

A comment on Partitioned consensus and its impact on Spanner’s latency. Parent comment

Re:Partitioned consensus and its impact on Spanner’s latency by jamesblonde on Friday December 14, 2018 @03:07PM (#18684180)
I should have been more clear - Calvin serializes cross-partition transactions. NDB does not. There is a Transaction Coordinator (TC) on every node. TCs can execute cross-partition transactions in parallel, but programmers need to write "lock-aware" programs (more late). TCs can fail-over if one fails - so, it blocks for just a few seconds (Transaction inactive timeouts are typically just a couple of seconds). There are no leaders in each partition, because every node is a potential TC. There are primary TCs for each partition, but failure does not require a leader election.

NDB has "lock-aware" programming - you don't get "global consensus". You decide, as a programmer, that this row could be accessed concurrently by another process, so you lock it, with either a read of write lock. Linearizability is easily implemented by acquiring a lock on a well-known row, but, of course, kills scalability.

In our Usenix FAST paper on HopsFS on Spotify's Hadoop workload, we had 1m ops/sec on HDFS, which was about 10m ops/sec on NDB. We ran out of hardware. There are workloads that big. [edited for clarity]

[ Parent | on HN ]
Re:Partitioned consensus and its impact on Spanner’s latency by abadid on Friday December 14, 2018 @03:12PM (#18684230)
Please, please read the Calvin paper http://www.cs.umd.edu/~abadi/papers/calvin-sigmod12.pdf. The assumption that Calvin serializes cross-partition transactions is a common misunderstanding is 100% inaccurate. The paper shows how Calvin gets better parallelism on cross-partition transactions than traditional systems.
[ Parent | on HN ]
Re:Partitioned consensus and its impact on Spanner’s latency by jamesblonde on Friday December 14, 2018 @03:16PM (#18684275)
Ok, sorry about that if it wasn't correct. But are you still not globally ordering every cross-partition transaction - "every scheduler to piece together its own view of a global transaction order by interleaving (in a deterministic, round-robin manner) all sequencers’ batches for that epoch". Even if the sequencers are distributed and execute the transactions in parallel, they need to agree on a total order. This contrast with the lock-aware programming model in NDB, where programmers can allow cross-partition transactions to proceed immediately in parallel if they are sure that they don't conflict.
[ Parent | on HN ]
Re:Partitioned consensus and its impact on Spanner’s latency by abadid on Friday December 14, 2018 @03:46PM (#18684551)
Yes, that's what I refer to as "unified consensus". Batching helps to overcome the scalability challenges (i.e. order batches rather than each individual xact). I talk about this briefly at: http://dbmsmusings.blogspot.com/2018/09/newsql-database-syst...
[ Parent | on HN ]