Re:Just 11% of 53 cancer research papers were reproducible
by gwern on Thursday May 09, 2013 @11:23PM
(#5684402)
> My guess is that if you included all test results, that reject the null hypothesis or not, and are published or not, then the reproducibility rate would be closer to 90%-95%.
No; you need to bring in power, not just assume an alpha of 0.05 and a base-rate of real results of 20%. (Consider the recent neuroscience paper estimating the experiments average a power of like 0.3...)
My guess is that if you included all test results, that reject the null hypothesis or not, and are published or not, then the reproducibility rate would be closer to 90%-95%.