Re:You Should Downvote Contrarian Anecdotes
by stephencanon on Monday June 18, 2012 @08:03AM
(#4125992)
95% confidence would be fine (or very nearly fine) if a requirement for funding/publication would be that all data, including negative results, be published as well.
[ Parent
| on HN ]
That would only be true if
a) all data, everywhere in the world, including negative results, was published regardless of funding/publication.
b) someone actually looked at that data, normalized it, and used it to assess the real significance of every result, in a sane manner (e.g. by using a bayesian inference with some reasonably behaving universal prior).
Neither a, b will ever happen, and both are essential.
(note: publication of all data is not a sufficient requirement: if 20 independent labs each do the same random experiment, one of them is expected to have a 95% confidence, and when they publish all their data, it consists of that one experiment that seems legit. This _will_ and _already does_ happen by chance)