What informs your priors?
That is the core issue with practical applications of Bayesian statistics. Especially because it is a totally new parameter that people can complain about. At best this leads to rampant bikeshedding, at worst we get prior-hacking as opposed to p-hacking.
What I wonder is, given a system with perverse incentives, won't people find a way to abuse Bayesian statistics?
I'm sure they would! But there's a pretty strong argument it's a step in the right direction. Actually, this link makes a more modest proposal that papers should report likelihoods rather than p-values. This avoids reported results results depending on priors (which perhaps we don't trust authors to choose well), though a reader can easily impose themselves if they want to.
https://arbital.com/p/likelihoods_not_pvalues/
You still have the option to muck with things by choosing your hypothesis class in a bad way-- nothing can really replace publishing data!
Probably.. but with Bayesian methods at your model/data updates its priors, rather than you effectively embedding your prior beliefs into your models via selectively choosing tests that support them.
It's very easy to let that slip into post hoc justification of your priors.
In my view, "prior" may be a misnomer. There is nothing that I'm aware of in Bayes' theorem to suggest that you have to formulate your priors before gathering or analyzing your data. I would describe priors as constraints that are included in an analysis, to narrow the results based on additional information that you're aware of. Bayes' theorem mainly provides a framework for computing what happens when you do that.
Maybe the prior doesn’t need to be formulated _before_ getting the data, but it needs to be _independent_ of the data.
Why become Bayesian? The rational actor theorems that drove that in the '60's turn out not to apply if you use a class of priors instead of a single prior, and single priors are overspecific to capture belief in most cases.
Better to go all the way down to decision theory.
Re:Ways to fix statistics
by Bromskloss on Saturday December 02, 2017 @01:34AM
(#15830569)
Do you have some fuller description of, or some links to, the work you are referring to?
Also, would decision theory replace Bayesian inference? Would it not rather _use_ it?
Seriously though, I am inclined towards approaches, such as pre-registration, which limit the number of researcher degrees of freedom in analysis. It's not necessarily that statistics are broken. It's that the system incentivizes researchers to break the assumptions underlying these statistical tests.