Re:p-Hacking and False Discovery in A/B Testing
by joshuamorton on Sunday July 22, 2018 @02:33AM
(#17585543)
p-hacking has nothing to do with any specific significance level. You can p-hack at a significance of .5 or .05 or .000005. A better definition would be
>p-hacking is a set of related techniques, whereby clever experimental design and cherry picking of data can generate results that falsely appear statistically significant.
There are a few important differences here:
1. The effect is not statistically significant. In fact often, there is no effect at all.
2. There is no mention of a specific significance level.
>p-hacking is a set of related techniques, whereby clever experimental design and cherry picking of data can generate results that falsely appear statistically significant.
There are a few important differences here:
1. The effect is not statistically significant. In fact often, there is no effect at all.
2. There is no mention of a specific significance level.
Those are both important.