Re:Reproducibility Initiative gets $1.3M grant to validate 5...
by Eliezer on Wednesday October 16, 2013 @05:17PM
(#6562208)
> I think that in general you're right
Oh, dear. This is very much not a good sign, since answer to grandparent's question is "no" as pointed out above and anyone who knows statistics should see that immediately. (Checks Science Exchange site.) Okay, Bilal Mahmood is "customer support and outreach" so this is not necessarily fatal. Still worrisome.
I think that in general you're right, with the assumption that p=0.05. However, each of the studies being replicated is composed of many intricate experiments. We will be reproducing most of those experiments, so while the odds of a single validation experiment failing are certainly not zero, I think the chances of an entire paper's worth of reproduction experiments being false negatives is much less likely. All of the protocols and reagents used, and the data from the replication studies, will be publicly available when they are complete, so that everyone can see what we did and how we did it.