News for hackers. Stuff that matters.
Re:Reproducibility Initiative gets $1.3M grant to validate 5... by bmahmood on Wednesday October 16, 2013 @04:27PM (#6561964)
This is Bilal from Science Exchange, and I help with the Reproducibility Intiative as well.

I think that in general you're right, with the assumption that p=0.05. However, each of the studies being replicated is composed of many intricate experiments. We will be reproducing most of those experiments, so while the odds of a single validation experiment failing are certainly not zero, I think the chances of an entire paper's worth of reproduction experiments being false negatives is much less likely. All of the protocols and reagents used, and the data from the replication studies, will be publicly available when they are complete, so that everyone can see what we did and how we did it.

Re:Reproducibility Initiative gets $1.3M grant to validate 5... by Eliezer on Wednesday October 16, 2013 @05:17PM (#6562208)
> I think that in general you're right

Oh, dear. This is very much not a good sign, since answer to grandparent's question is "no" as pointed out above and anyone who knows statistics should see that immediately. (Checks Science Exchange site.) Okay, Bilal Mahmood is "customer support and outreach" so this is not necessarily fatal. Still worrisome.