Re:Research software code is likely to remain a tangled mess
by tylermw on Monday February 22, 2021 @01:00PM
(#26227798)
> good experiments change a single variable at a time
Although this is a tangent from the above conversation, this isn't actually true: well-designed experiments can indeed change multiple variables at the same time. There's an entire field of statistics dedicated to experimental design (google "factorial designs" for more information). One-factor-at-a-time (OFAT) experiments are often the least efficient method of running experiments, although they are conceptually simple.
See the following article for a discussion: http://www.engr.mun.ca/~llye/czitrom.pdf
And you say it yourself: good experiments change a single variable at a time. So how do you check that a series of potential improvements that you are making is sound?