Simplicity is a nice dream. The realities of research are very often stacked against it.
I've seen research groups drown in their legacy code base.
The issue of juggling too many balls you describe is one you only have to begin with because the state of the art implementations are so shoddy to begin with.
Research suffers as much as everybody else from feature creep. Good experiments keep the number of new variables low.
And you say it yourself: good experiments change a single variable at a time. So how do you check that a series of potential improvements that you are making is sound?
Although this is a tangent from the above conversation, this isn't actually true: well-designed experiments can indeed change multiple variables at the same time. There's an entire field of statistics dedicated to experimental design (google "factorial designs" for more information). One-factor-at-a-time (OFAT) experiments are often the least efficient method of running experiments, although they are conceptually simple.
See the following article for a discussion: http://www.engr.mun.ca/~llye/czitrom.pdf
A Google search makes it look like Julia has a mechanism where you can extent the sets of overloads of a function or method outside the original module. The terminology is different (functions have methods instead of overloads in their speak). I don't see how that feature solves the problem in practice.
Check https://stackoverflow.com/questions/1801216/what-is-the-diff... and https://www.youtube.com/watch?v=kc9HwsxE1OY
Note that Simple != Easy or Naive
Hardcoded structures is potentially exactly the kind of simplicity needed.
What's not simple is a general "this solves everything and beyond" code-base with every imaginable feature and legacy capability.
Ironically this is also what occams razor would demand from good Science, so you'd have a win win scenario, where you both create good software and good research, because you focus on the simplest most minimal approach that could possibly work.