In cases where there is significant differences in the maturity between theory and experiment, you either end up with theorists playing games with math, making up tons of unconstrained theories (string theory, for instance), or in the other direction you end up with experimentalists measuring anything and everything they can imagine, half of which is probably not theoretically enlightening, waiting for theorists to constrain their space of possible experiments to potentially fruitful paths.
Obviously this is harder for fields that don’t have models as robust as we do in physics, but I’d guess the same phenomenon happens everywhere.
So, where is the progress coming from?
Your argument that "there must be a robust and explanatory theory that goes along with experimental results" helps make sense of TFA, which otherwise presents as a paradox, or a ruse to excuse shoddy practices.
The presence of a viable theory is a useful heuristic for sorting useful experimental practice from less useful. At least, I imagine it is. How can we know for sure?
Has anyone published a meta analysis that checks for this?
And has that meta analysis been independently reproduced?
Nonetheless, reproducibility is a really nice property. You have to be incredibly careful with your theory if it can't be justified.
Scientific results are what makes science universal; since they can be reproduced everywhere by anyone.
A variety of causes have been proposed, such as abuse / ignorance of statistics, bad experimental design, and even outright chicanery.
In your view? Reproducibility is essential to scientific progress because it defines what science is. It's the backbone of science. Hypothesis,testing/experiments. It's what separates science from math, arts and pseudosciences like social "sciences".
> I'm sure if you went back through the literature on something like electromagnetism, you would find results that fail to replicate, yet the theory of electromagnetism if applied properly is remarkably robust.
How would it be "robust" if it couldn't be replicated? How would the theory of electromagnetism be robust if it failed to produce replicable results?
> Scientific results can be strengthened by replication
No. It can only be a "scientific result" if it can be replicated/tested. It's definitional.
> Discovery of a psychological "effect" is perfectly scientific and interesting
No it isn't if it can't be test/replicated/reproduced. It's not science. It's something else altogether.
> And if those sciences are also plagued by irreproducibility, then they may embrace scientific methodology without producing a useful scientific knowledge base.
There is no "scientific methodology" without testability/replication/reproducibility.
All science is based on hypothesis-testing.
I read that in the century after Newton, the French Academy offered a prize for evidence of the failure of Newton's laws. They gave out the prize dozens of times, yet Newtonian physics kept getting stronger and stronger. Eventually they stopped giving out the prize. Many of the contradictions apparently had to do with the lunar orbit, which was poorly understood.
Deals with some of your points.
Scientific results can be strengthened by replication, but that's just one thing, and if that's all you do, then you end up with a science that does nothing but generate independent factoids.
On the other hand, robust science tends to look at a particular phenomenon from many different angles, and manages to connect multiple results together into a framework that can survive retracting individual studies without collapsing. This is how electromagnetism developed.
Sciences that are in the factoid phase are not necessarily junk. Discovery of a psychological "effect" is perfectly scientific and interesting. But some sciences have barely progressed beyond the factoid phase. And if those sciences are also plagued by irreproducibility, then they may embrace scientific methodology without producing a useful scientific knowledge base.