News for hackers. Stuff that matters.
Re:P-Hacking in Startups by kgwgk on Sunday June 22, 2025 @11:40AM (#44347800)
There are a few good explanations already (also less good and very bad) so I give a simple example:

You throw a coin five times and I predict the result correctly each time.

#1 You say that I have precognition powers, because the probability that I don’t is less than 5%

#2 You say that I have precognition powers, because if I didn’t the probability that I would have got the outcomes right is less than 5%

#2 is a bad logical conclusion but it’s based on the right interpretation (while #1 is completely wrong): it’s more likely that I was lucky because precognition is very implausible to start with.

Re:P-Hacking in Startups by jonahx on Sunday June 22, 2025 @06:30PM (#44350840)
Dead on again.

What this and your other comment make clear is that once you start talking about the probability that X is true, especially in the context of hypothesis testing, you've moved (usually unwittingly) into a Bayesian framing, and you better make your priors explicit.