Re:Just 11% of 53 cancer research papers were reproducible
by codex on Thursday May 09, 2013 @06:13PM
(#5683344)
This can be explained without bringing in accusations of fraud or incompetence [1].
Basically, if you have a bazillion experiments running and only publish the results which are significant, you still have a huge predictability problem, because most of those statistically significant results will be due to chance alone.
For example, assume that only 1% of cancer experiments produce something valid and interesting, and that standard for statistical significance is the industry standard 95%, or 1 in 20. If 100,000 hypothesis are tested, you'll get 100,000 * 1% = 1000 valid results and 100,000 * 5% = 5000 results from chance alone. In this example, only 16% of the published results will be reproducible, because only 1000 / 6000 are valid.
Re:Just 11% of 53 cancer research papers were reproducible
by dkural on Friday May 10, 2013 @04:11AM
(#5685147)
this basic fact is understood by scientists and children alike. it is still either incompetence (lack of middle school statistics) or fraud (ignoring the above fact).
Basically, if you have a bazillion experiments running and only publish the results which are significant, you still have a huge predictability problem, because most of those statistically significant results will be due to chance alone.
For example, assume that only 1% of cancer experiments produce something valid and interesting, and that standard for statistical significance is the industry standard 95%, or 1 in 20. If 100,000 hypothesis are tested, you'll get 100,000 * 1% = 1000 valid results and 100,000 * 5% = 5000 results from chance alone. In this example, only 16% of the published results will be reproducible, because only 1000 / 6000 are valid.
[1] http://sciencehouse.wordpress.com/2009/05/08/why-most-publis...