Re:FoundationDB: A Distributed Key-Value Store
by falsandtru on Monday July 03, 2023 @02:24PM
(#36576946)
Those tests are not the implementations of the tests, just specifying the test case. But I found the implementations. I am not sure if this is all of the simulation tests, but it seems to cover the major cases.
https://github.com/apple/foundationdb/tree/main/fdbserver/wo...
> Someone could probably integrate this functionality into a Kubernetes operator or something so that outside engineers could run large scale simulations reliably. But it is really expensive and CPU/compute intense, no matter how you go about it.
Maybe this.
Re:FoundationDB: A Distributed Key-Value Store
by aseipp on Monday July 03, 2023 @02:47PM
(#36577261)
Yeah, that's basically an actually good implementation of the pile of crap that I threw together several years ago while writing a few patches. :)
And yes, I linked to the spec files because there actually isn't that much test code written in Flow I feel; the high-level specs in the .txt files can be mixed and matched so much to create a lot of variety from some small number of primitives, so that's really where all the good stuff is. Implementation vs interface, and all that.
However, they (at least at the time most of the developers were at Apple, many have now moved to Snowflake and the Apple team has grown a little I think) haven't released or integrated their nightly cluster and performance testing systems into open, nor have they integrated them with GitHub Actions or Nightly runs or anything. So while Apple or Snowflake integrates changes across hundreds of concurrent fuzzing simulations on whatever platforms they have, if you write patches yourself, you're stuck with long simulation runs. Maybe that's changed; I haven't kept up since the 7.0 series.
In practice if you write patches and they accept them, they will just do the testing in their runs for you, on a cluster far larger than what you could have.
Someone could probably integrate this functionality into a Kubernetes operator or something so that outside engineers could run large scale simulations reliably. But it is really expensive and CPU compute intensive no matter how you go about it.
[1] https://forums.foundationdb.org/t/how-to-use-foundationdb-un...
[2] https://github.com/apple/foundationdb/tree/main/tests