Re:Likelihood functions, p-values, and the replication crisis
by CrazyStat on Tuesday March 05, 2024 @08:37PM
(#39611267)
Well, I'm discarding whole experiments. Each experiment is reported in full, I just don't publish all of them. No scientist ever does.
His claim about likelihoods being immune to p-hacking is far too strong.
> With p-values, sometimes even honest experiments end up with misleading conclusions. It takes a decent amount of complex understanding to perform properly, while something like gathering data and doing a conditional probability calculation has presumably less risk of honest mistakes
Maybe. I've seen an experienced Bayesian statistician who had published a paper about Lindley's paradox fall prey to Lindley's paradox and publish a misleading conclusion as a result. Bayesian analysis also has some pitfalls. Name withheld to protect the innocent.
In any case he does mention it won't fix all the problems but that it can clean up some of them and make addressing other problems a bit more straightforward. With p-values, sometimes even honest experiments end up with misleading conclusions. It takes a decent amount of complex understanding to perform properly, while something like gathering data and doing a conditional probability calculation has presumably less risk of honest mistakes