Re:It’s not just p=0.048 vs. p=0.052
by bonoboTP on Wednesday September 11, 2019 @07:48PM
(#20946002)
> The probability of getting a p-value below 0.05 is 5%, the probability of getting a p-value below 0.25 is 25%, the probability of getting a p-value below 1 is 100%.
But getting a p value between 0.82 and 0.87 is also just 5% probability, so could also be seen as a probably-won't-happen-by-chance event. Sure it's clear most of the time which 5% is the significant one,but not always, for example paradoxically the same result can be significant or not, depending on what your intended stopping criterion was, even if you happened to have to stop at the same time using either criterion. This is because the two variants would lead to declaring a different 5% portion of the null's possible outcomes as the significant part.
But getting a p value between 0.82 and 0.87 is also just 5% probability, so could also be seen as a probably-won't-happen-by-chance event. Sure it's clear most of the time which 5% is the significant one,but not always, for example paradoxically the same result can be significant or not, depending on what your intended stopping criterion was, even if you happened to have to stop at the same time using either criterion. This is because the two variants would lead to declaring a different 5% portion of the null's possible outcomes as the significant part.