Re:It’s not just p=0.048 vs. p=0.052
by penagwin on Wednesday September 11, 2019 @09:27AM
(#20939880)
My understanding (Someone please correct me if I'm wrong) is that while the values can range from limx->0 and limx->1 , the actual distribution will be skewed in cases where it is statistically significant. E.g. If you repeat the experiment 100x you could find a p=.999999 but more likely they'll be close to .05 (or whatever alpha you choose)
If the null hypothesis is true the p-value is distributed uniformly in [0 1] (at least ignoring discrete data, composite hypothesis and other cases which are not easy).
The distribution will be different when then true hypothesis is not true (but you may also get non-significant results even if the null hypothesis is not true).
I’m not sure if that’s what you mean by “the actual distribution will be skewed in cases where it is statistically significant.”
Re:It’s not just p=0.048 vs. p=0.052
by penagwin on Wednesday September 11, 2019 @01:02PM
(#20942301)
Sorry I was definitely kinda vague - I meant when the alternative hypothesis is true the distribution should be skewed towards 0.
Re:It’s not just p=0.048 vs. p=0.052
by lonelappde on Wednesday September 11, 2019 @08:57AM
(#20939567)
The point is that you need to replicate experiments and not cherry pick the one with the highest p-value.
At which point, what is even the point of this statement?