News for hackers. Stuff that matters.
Re:Statisticians want to abandon science’s standard measure ... by SquishyPanda23 on Friday September 06, 2019 @11:40AM (#20896624)
The problem isn't that the wording is confusing, it's that p(x|H0) isn't a very useful thing to compute.

Everybody wants to compute p(H|x), the probility of a scientific hypothesis given the data. People want to do this so badly that they can't help interpreting the p-value that way.

You can actually compute p(H|x) if you use Bayesian stats.

Re:Statisticians want to abandon science’s standard measure ... by CrazyStat on Friday September 06, 2019 @11:55AM (#20896776)
With some caveats: the value you get for P(H|x) might be very sensitive to the priors you choose, and most people are not thoughtful enough about this.
Re:Statisticians want to abandon science’s standard measure ... by SquishyPanda23 on Friday September 06, 2019 @03:11PM (#20898816)
Yes that is true, but there is a sense in which that just makes explicit a problem that already exists.
Re:Statisticians want to abandon science’s standard measure ... by andrewla on Friday September 06, 2019 @02:23PM (#20898286)
> You can actually compute p(H|x) if you use Bayesian stats.

You can't "compute" it for any useful meaning of the word "compute". You can estimate it intuitively, or you can try to look at how many pre-registered unpublished studies or null results have been published. Otherwise there's no way to even consider getting a grasp of what P(H) would be.

Re:Statisticians want to abandon science’s standard measure ... by hervature on Friday September 06, 2019 @04:01PM (#20899367)
> Otherwise there's no way to even consider getting a grasp of what P(H) would be.

That's called picking a prior. Usually, pick the prior that maximizes the entropy on the given interval (zero prior knowledge).

For those reading and confused. Computing: p(H|x) = p(x|H)p(H)/p(x)

requires p(H) which the parent is suggesting is impossible to grasp.

If we want to know the probability that a coin is biased, we can assign probabilities to each hypothesis. For some people, p(bias=1/2)=1 (Dirac distribution), others might argue that p(bias=x)=1 for x in [0,1] (uniform distribution). Others might argue its some Beta function centered around 1/2. I believe what the parent is suggesting is that choosing which original belief we have in the system is a matter of philosophy, not computation.

Re:Statisticians want to abandon science’s standard measure ... by pfortuny on Friday September 06, 2019 @01:22PM (#20897616)
Wait wait wait: you need the priors, also. Where do they come from?
Re:Statisticians want to abandon science’s standard measure ... by SquishyPanda23 on Friday September 06, 2019 @02:48PM (#20898601)
That's definitely the hard part of switching to Bayesian methods. But the problem doesn't go away if you use p values, it just gets ignored.

Typically people doing research have prior information about what they are researching. E.g. previous studies have found effect sizes to be in some interval.

You can fallback on an uninformative prior or other tricks if you really want to model the idea that you know literally nothing about what you're researching. But that should be very rare.

Re:Statisticians want to abandon science’s standard measure ... by NohatCoder on Friday September 06, 2019 @02:59PM (#20898704)
You just make priors up, just like the 0.05 limit is made up. Making up a prior is basically asking the question: How likely does it seem that X has Y effect? Without having any data specific to that question.

Our 0.05 p-value limit is effectively just answering that question with a fixed 5%, no matter how ridiculous the proposition is.