Re:Replacing a silently failing disk in a ZFS pool
by kbenson on Wednesday July 03, 2019 @03:30PM
(#20347825)
On this topic (and related in many ways) is Bryan Cantrill's talk Zebras All the way Down.[1]
The gist is that in medicine there's the concept of diseases that are zebras and horses, rare and common, and the saying goes "when you hear hoof beats think horses not Zebras". Unlike medicine, in software most the easy problems have already been abstracted away or made trivial through our software stacks, so often what's left are Zebras. What used to be the exceedingly rare bugs now account for a large proportion of the odd behavior we see in very robust tech stacks, because the easy bugs have been hunted down and eliminated already.
Re:Replacing a silently failing disk in a ZFS pool
by brandmeyer on Wednesday July 03, 2019 @03:55PM
(#20348106)
“Every method you use to prevent or find bugs leaves a residue of subtler bugs against which those methods are ineffectual.”
Boris Beizer, Software Testing Techniques. Second edition. 1990
> In a typical case, if you don't cook pork well enough, you digest live tapeworm larvae. They've got these little hooks, they grab onto your bowel, they live, they grow up, they reproduce. Reproduce? There's only one lesion, and it's nowhere near her bowel. That's because this is not a typical case.
New professionals always expect things to be similar to what they studied. What they learn: the books and lectures often covered the "most common" situations, but all those edge cases and weird scenarios come up on a daily basis.
[1]: https://www.springfieldspringfield.co.uk/view_episode_script...