Re:YDB – An open-source Distributed SQL Database
by fomichev3000 on Tuesday April 19, 2022 @04:23PM
(#31088526)
Each tablet gather a quorum of answers from members of so called BlobStorage group. BlobStorage group is a number of so called VDisks (virtual disk), all VDisks run on different nodes (even on different fail domain like racks, AZs). VDisk stores its data on physical device, i.e. PDisk.
Re:YDB – An open-source Distributed SQL Database
by anikuni on Wednesday April 20, 2022 @10:50AM
(#31097765)
From my past experience, Datomic uses the same approach, ie, multiple reader nodes and a single transact node. However, it's much more locked in with AWS, as it uses Dynamo and S3 for backing (maybe others as well?).
Re:YDB – An open-source Distributed SQL Database
by jeffbee on Tuesday April 19, 2022 @11:52AM
(#31084809)
Assigning leaders is trivial with something like zookeeper. But in this case it appears that the leader metadata is stored in a table of the database itself, which raises questions of operability if those tablets are unavailable.
Re:YDB – An open-source Distributed SQL Database
by fomichev3000 on Tuesday April 19, 2022 @04:26PM
(#31088564)
YDB doesn't use Zookeeper. The system is built of tablets, every tablet implements distributed consensus algorithm. There are different types of tablets in the system, say SchemeShard is tablet that stores metadata, table schema for instance. DataShard stores table partition data.
Re:YDB – An open-source Distributed SQL Database
by asadawadia on Tuesday April 19, 2022 @07:26PM
(#31090711)
Which consensus algorithm? I don't see raft in the codebase
Re:YDB – An open-source Distributed SQL Database
by fomichev3000 on Wednesday April 20, 2022 @05:11AM
(#31094410)
We have consensus protocol over shared log.
Some details you can find in a power point presentation at this page https://2019.hydraconf.com/2019/talks/muxomfgqembsb3st7i3ci/
Assuming yes this approach, I think, is under utilized and is pretty similar to how Apache Pulsar works (my day job),but I am not sure how many distributed RDBMS have tried it out, will be cool to see how it evolves! It isn't clear how they ensure the assignment of a tablet to only a single compute node, but I think that is an easier problem relative to distributed consensus at the storage tier.