Not Even Scientists Can Easily Explain P-values

Re:Not Even Scientists Can Easily Explain P-values by bweitzman on Wednesday November 25, 2015 @12:02PM (#10628118)

I think the coin example makes sense but is not a good example in this case. You will never be able to determine the validity of a coin just by flipping it. Any sequence of flips is technically valid but possibly just very rare for a fair coin.

The p-value won't tell you if the coin is fair or not, but it can tell you the probability that the coin is fair.

[ Parent | on HN ]

Re:Not Even Scientists Can Easily Explain P-values by skybrian on Wednesday November 25, 2015 @12:15PM (#10628213)

If the sequence is rare enough then we do, in practice, conclude that the coin is unfair. If you can't make that conclusion then science doesn't work; no observations will convince you.

[ Parent | on HN ]

Re:Not Even Scientists Can Easily Explain P-values by bweitzman on Wednesday November 25, 2015 @12:24PM (#10628281)

Of course. If you flip the coin 100 times and get all heads than you are safe to call the coin unfair, even though you would expect to see that result (1/2)^100 and thus would be wrong once in a while.

I think that's sort of the point that the article is making actually. That high probability does not imply truth. There are other non probabilistic ways to verify that coin is unfair, for example by looking at the density throughout the coin.

[ Parent | on HN ]

Re:Not Even Scientists Can Easily Explain P-values by TeMPOraL on Wednesday November 25, 2015 @12:43PM (#10628391)

Observations do not imply truth. 0 and 1 are not probabilities, and in real life you can't prove something using observations in a logical sense. Real world runs on probabilities, not boolean logic.

Even if you look at the density of the metal throughout the coin, there's still a chance I've altered your device to report the coin is fair. Or a passing microsingularity decided to play games with the scanning beam. Or you're just imagining the whole thing.

That's not to say one should despair that the world is unknowable. One only has to get used to the fact that, in practice, "true" just means "extremely, extremely likely".

[ Parent | on HN ]

Re:Not Even Scientists Can Easily Explain P-values by bweitzman on Wednesday November 25, 2015 @12:53PM (#10628439)

Sure, but there is a difference on the order of magnitudes between the probability that a fair coin will come up heads 100 times in a row and the probability that a microsingularity will come along and bias your results.

But yeah truth is tricky.

[ Parent | on HN ]

Re:Not Even Scientists Can Easily Explain P-values by tedsanders on Wednesday November 25, 2015 @02:18PM (#10628970)

No no no. A p-value cannot tell you the probability a coin is fair. This is exactly the misconception that makes p-values a bad tool.

Suppose I have a coin and flip HHHHH. Can you tell me the probability the coin is fair? No, it's fundamentally unknowable. We can say that a fair coin would have a 3% chance of flipping HHHHH (the p-value), but we can't say with what probability our coin is fair.

[ Parent | on HN ]