About 40% of economics experiments fail replication survey (2016)

Re:About 40% of economics experiments fail replication surve... by viraptor on Monday September 17, 2018 @05:55AM (#18004481)

While you're right about the 5%-or-less chance, it's still a 5% chance we're aiming for. If we end up with 40% over a large number of papers, something went wrong.

Not publishing the negative results shouldn't affect this number. They're not included in the 5% chance in the first place.

[ Parent | on HN ]

Re:About 40% of economics experiments fail replication surve... by evandijk70 on Monday September 17, 2018 @06:23AM (#18004605)

This is a common misunderstanding of p-values.

There is a 5% chance of observing the effect, if the effect is not there.

I think the difference is best illustrated by an example:

If you have 20 researchers investigating a hypothesis that turns out to be false 1 of them will report evidence for a false hypothesis, which will likely not replicate. Thus, if for every true hypotheses investigated, 20 false hypotheses are investigated, and the 19 researchers with false hypotheses do not report their result, that means that 1/2 of the reported positive results will not replicate.

[ Parent | on HN ]