> We produce shit studies with small sample sizes and dodgy methodologies all the time. They just don't add to the pool of human knowledge.

They actually do. A crappy study with p=.70 isn't enough to justify changing the standard treatment for a disease but it is enough to justify doing the more comprehensive study to see if the result still holds.

We have a choice between doing ten comprehensive studies only to prove that nine of the things didn't work, or using the same resources to do a hundred crappy studies and then repeat the five with the most promising results using the more robust methodology and discover that three of them still hold.

You confuse p-value with methodology. P-value is entirely a product of sample size and effect size. For a given effect size, the cheaper I want the study to be, the smaller I make the sample, and the less likely I am to reach a significant p-value.

The problem is when the methodology itself is broken. For instance, using case-control methodologies on things with small-to-moderate effect sizes. Past studies comparing such studies with eventual RCTs showed that the case control studies gave absolutely no indication of what the RCTs would show as a consensus: it was entirely random.

There is no amount of money, small or large, worth throwing at studies that shift our prior probability to posterior by 0 units.