It’s not just p=0.048 vs. p=0.052

Re:It’s not just p=0.048 vs. p=0.052 by mehrdadn on Wednesday September 11, 2019 @03:46PM (#20943908)

So I don't have (or know) R, but I do have access to Mathematica, and this is most definitely not giving me a uniform distribution (how could it?!):

  << HypothesisTesting`
  With[{n = 10000000, dist = NormalDistribution[]},
   Histogram[Last[NormalPValue[RandomVariate[dist, n] - RandomVariate[dist, n]]], 500]]

Why is your x variable though? If H0 is true shouldn't your x be fixed?

I think if you remove the subtraction though then you do get a uniform distribution -- in which case I see what the claim is, yeah. Wasn't really clear to me earlier but indeed, getting p = 5% means you have a 5% chance of getting observations that extreme, so I guess it is uniformly distributed!

[ Parent | on HN ]

Re:It’s not just p=0.048 vs. p=0.052 by dash2 on Friday September 20, 2019 @04:59AM (#21024580)

You'd get the same result whether you test the hypothesis y > x (with both random variables) or y > 0. Under the null, the p-value is uniform on [0, 1], whatever that null is. It follows from the definition of a p value.

(Disclaimer: I am not a real statistician....)

[ Parent | on HN ]