Re:Jepsen: MongoDB 4.2.6
by throwaway_pdp09 on Sunday May 24, 2020 @12:22PM
(#23292555)
I guess you've answered my question, but to be clear, you do not instrument/analyse the code, you treat it as a black box which you hammer on externally, is that right?
Pretty much, yeah. There are some cases where Jepsen reaches into the guts of a database or lies to it via LD_PRELOAD shims, but generally these are Just Plain Old Binaries provided by vendors; no instrumentation required.
Jepsen draws inspiration from a long line of work on property-based testing, especially Quickcheck & co. It also draws on roughly 10 years of experience building & running distributed systems in production. A lot of Jepsen I invented from whole cloth, but some of the checkers in Jepsen are derived from specific research papers, like work by Wing, Gong, and Howe on linearizability checking.
Then it's just... a lot of thinking, experimenting, and writing. Jepsen's the product of ~6 years of full-time work. Elle, the system which detected the anomalies in this report, was a research project I've been puzzling over for roughly two years.
I write the Jepsen series, and open-source all of the code for these tests, partly as a resource so that other people can learn to do this same kind of work. :-)