Re:Objective Bayesian Hypothesis Testing
by vcdimension on Friday August 23, 2024 @01:33AM
(#41326426)
This article is very interesting and informative, however it's a bit ironic that an article about misinterpretations of the meaning of the p-value, misinterprets the misinterpretation; in the first blue box it's clear that Bernstein is interpreting the p-value as the probability of randomly rejecting the null (which is what you do when you get something statistically significant) yet in the text following that they say he's interpreting it as the probability of the null.
Bernsteins mistake is that he appears to interpret it as an unconditional probability rather than a conditional one (correct interpretation; p-value = Prob(rejecting the null when the null is true)).
[ Parent
| on HN ]
Yes I had the same issue. But the wording "there is a < 5% probability that an outcome was the result of chance" is in fact problematic since many readers will go on to conclude "hence a >95% probability that the outcome was not the result of chance", so it is easier to misinterpret than the technical definition P( Observation | H_0 ).
In courses I will typically use wordings like "If there was truly no association, then the probability of getting an observation like this is <5%".
> correct interpretation; p-value = Prob(rejecting the null when the null is true)
This is also not quite correct. The p-value is the probability of falsely rejecting the null due to sampling error. It is quiet on all other errors that are frequently committed.
The real probability of falsely rejecting the null starts at 15 % thanks to mathematical slip-ups alone: https://two-wrongs.com/the-lying-p-value