News for hackers. Stuff that matters.
Re:It’s not just p=0.048 vs. p=0.052 by pacbard on Wednesday September 11, 2019 @07:39AM (#20938870)
He is coming at that conclusion from a Bayesian point of view to statistics. He is seeing the p-value as a random variable that can take values from 0 to 1 and follows some distribution. Under these hypotheses, observing a p-value of 0.20 and 0.005 is completely reasonable even if unlikely. Those are just two draws from a random variable.

Edit. Under Bayesian statistics testing the null hypothesis is a moot point as it becomes possible to directly model the distribution of the possible effects. Thinking of it as being able to look at a picture of something (the p-value) vs looking at a movie of it (the distribution of the effects).

Re:It’s not just p=0.048 vs. p=0.052 by JustFinishedBSG on Wednesday September 11, 2019 @07:44AM (#20938908)
That's not even a bayesian point of view, even in a frequentist setting the p-value is a conditional probability, conditioned on the dataset.
Re:It’s not just p=0.048 vs. p=0.052 by pfortuny on Wednesday September 11, 2019 @07:42AM (#20938891)
What he says is (I gather) worse: those events are only separated be 1.1std deviations, which is little.
Re:It’s not just p=0.048 vs. p=0.052 by mehrdadn on Wednesday September 11, 2019 @07:56AM (#20938995)
I think that's what he's saying too, but what is that supposed to show? Is he arguing against some claim that every interval of 1 standard deviation is equally significant? Did anybody make this claim? So far as I know, nobody considers (say) a 6-sigma effect to be 6 times stronger than a 1-sigma effect...
Re:It’s not just p=0.048 vs. p=0.052 by kgwgk on Wednesday September 11, 2019 @01:44PM (#20942762)
I don’t get it either. It’s a trivial consequence of having a threshold: if we say two cities are “far” when they are at least 1000 miles away then Washington D.C. is not far from Jacksonville while Boston is far from Jacksonville, even though Boston is not far from Washington.
Re:It’s not just p=0.048 vs. p=0.052 by nerdponx on Wednesday September 11, 2019 @03:35PM (#20943830)
That's not quite the problem.

Let's assume you've already decided in advance what "far" means.

Without moving either city from its current location, the same experiment can give you "very far" and "very close" in identical replications.

Re:It’s not just p=0.048 vs. p=0.052 by kgwgk on Wednesday September 11, 2019 @04:16PM (#20944186)
I’m commenting on pfortuny’s (correct) interpretation “those events are only separated be 1.1std deviations, which is little” of the original post by Gelman (the difference between a significant result and a non-significant result may not be significant = a city which is far from Jacksonville and a city which is not far from Jacksonville may not be far from each other).
Re:It’s not just p=0.048 vs. p=0.052 by mehrdadn on Wednesday September 11, 2019 @07:43AM (#20938905)
I still don't get it. What p-value observations would not be "reasonable" here? It seems to me he's saying that anything between 0 and 1 is completely reasonable, which is a completely pointless statement as I see it.
Re:It’s not just p=0.048 vs. p=0.052 by lonelappde on Wednesday September 11, 2019 @08:56AM (#20939556)
His point is that the whole point of the field of statistics is that cherry picking results or getting lucky once is not proof of anything, unlike nonstatistical mathematics where one example is enough to justify a claim.
Re:It’s not just p=0.048 vs. p=0.052 by kgwgk on Wednesday September 11, 2019 @12:42PM (#20942088)
> He is seeing the p-value as a random variable that can take values from 0 to 1 and follows some distribution.

That is the frequentist point of view!