News for hackers. Stuff that matters.
Re:How FoundationDB works and why it works by zinodaur on Monday September 18, 2023 @10:48AM (#37556876)
If theres just one Sequencer, and every ReadVersion request to the Proxy eventually hits the Sequencer 1-1, how does the Sequencer not get crushed? Or is a scaling limit just "the number of ReadVersion requests a Sequencer machine can handle per second", which admittedly is a cheap request to respond to
Re:How FoundationDB works and why it works by lowbloodsugar on Monday September 18, 2023 @08:11PM (#37564041)
Yeah that seems like an untenable design choice. Was quite interested until I read that. Max TPS? and MTTR when sequence inevitably shits itself?
Re:How FoundationDB works and why it works by richieartoul on Monday September 18, 2023 @09:48PM (#37564746)
Replied above
Re:How FoundationDB works and why it works by foobiekr on Monday September 18, 2023 @10:31PM (#37565002)
You can trivially scale fdb to tens of millions of tx/sec for write-heavy workloads without a hardcore cluster for transactions of reasonable complexity (though with careful design on my part and the part of others for collisions to be unlikely).

MTTR on failure is seconds. Really, there's no system I've used that is as robust and performant as fdb and I include s3 in that list - s3, for example, _routinely_ has operations with orders of magnitude latency variance and huge, correlated spikes.

Re:How FoundationDB works and why it works by richieartoul on Monday September 18, 2023 @09:47PM (#37564740)
Requests to the sequencer are batched heavily. If the sequencer fails, the cluster goes through a recovery and will be unavailable for 2-3 seconds and then recover.
Re:How FoundationDB works and why it works by zinodaur on Tuesday September 19, 2023 @10:22AM (#37570374)
Good point about the batching! Any idea what kind of ReadVersion qps throughput you can get this way? And yeah, 2-3s unavailability seems fine.
Re:How FoundationDB works and why it works by whizzter on Tuesday September 19, 2023 @08:44AM (#37569153)
Not used FDB but reading the article and considering the semantics, it shouldn't matter too much if the Sequencer "just" distributes recent read-versions to the proxy frequently enough(unless that proxy has received a recent read-version "recently").

Worst case if there is heavy contention on the same keys then resolvers will eventually fail more transaction writes but for read-only transactions most applications should be fine with a slightly "old" version.

(Yes, all this will start to cause down if there is high key contention and many conflicts)

Re:How FoundationDB works and why it works by zinodaur on Tuesday September 19, 2023 @10:16AM (#37570285)
My understanding of ReadVersion is that the only point of calling it is to be able to read your own writes - so staleness wouldn't be good. There was another sibling comment that says the ReadVersion requests are batched up before hitting the sequencer, I could definitely believe that would work.
Re:How FoundationDB works and why it works by Dave_Rosenthal on Tuesday September 19, 2023 @05:18PM (#37576856)
Yeah, read versions are just there to make what's known as "external consistency" work. That is, if transaction B starts after transaction A commits, then B will see the effects of A.

The reason external consistency is nice is that if you change the database, as soon as you get a commit signal, you can tell any other client "hey, go check the database and see what I did" and they will see the changes. No worries about whether the changes have sync'ed yet or anything like that.

Re:How FoundationDB works and why it works by zinodaur on Thursday September 21, 2023 @11:53PM (#37607631)
We've been exploring ways of getting external consistency for our data model - but our Sequencers are different enough that I don't think we can use the FoundationDB approach.

We are trying out a version of what CockroachDB does - run transaction A possibly on stale data, spy on the things it reads, call our flavor of ReadVersion for the "shards" the transaction interacted with, and re-run transaction A with the data versions we got from ReadVersion. It will usually just return after the the first run, rarely requiring a second run, and in some pathological cases require infinite runs.