Re:Likelihood functions, p-values, and the replication crisis
by CrazyStat on Tuesday March 05, 2024 @08:37PM
(#39611267)
Well, I'm discarding whole experiments. Each experiment is reported in full, I just don't publish all of them. No scientist ever does.
His claim about likelihoods being immune to p-hacking is far too strong.
> With p-values, sometimes even honest experiments end up with misleading conclusions. It takes a decent amount of complex understanding to perform properly, while something like gathering data and doing a conditional probability calculation has presumably less risk of honest mistakes
Maybe. I've seen an experienced Bayesian statistician who had published a paper about Lindley's paradox fall prey to Lindley's paradox and publish a misleading conclusion as a result. Bayesian analysis also has some pitfalls. Name withheld to protect the innocent.
His claim about likelihoods being immune to p-hacking is far too strong.
> With p-values, sometimes even honest experiments end up with misleading conclusions. It takes a decent amount of complex understanding to perform properly, while something like gathering data and doing a conditional probability calculation has presumably less risk of honest mistakes
Maybe. I've seen an experienced Bayesian statistician who had published a paper about Lindley's paradox fall prey to Lindley's paradox and publish a misleading conclusion as a result. Bayesian analysis also has some pitfalls. Name withheld to protect the innocent.