News for hackers. Stuff that matters.
Re:Everything Is Correlated by limbicsystem on Thursday May 02, 2019 @04:37AM (#19805885)
It is true that, as Fisher points out, with enough samples you are almost guaranteed to reject the null hypothesis. That's why we tell students to consider both p values (which you could think of as a form of quality control on the dataset) and variance explained. Loftus and Loftus make the point nicely: p tells you if you have enough samples and any effect to consider, variance explained tells you if it's worth pursuing. Both are useful guides to a thoughtful analysis. In addition, I'd make a case for thinking about the scientific significance and importance of the hypothesis and the Bayesian prior. And to put a positive spin on this, given how easy it is to get small p values, big ones are pretty much a red flag to stop the analysis and go and do something more productive instead.
Re:Everything Is Correlated by nonbel on Thursday May 02, 2019 @11:13AM (#19808710)
> "It is true that, as Fisher points out, with enough samples you are almost guaranteed to reject the null hypothesis. "

Where does Fisher point this out?

> "That's why we tell students to consider both p values (which you could think of as a form of quality control on the dataset)"

How is this "quality control"? It just tells you whether your sample size was large enough to pass an arbitrary threshold...

Re:Everything Is Correlated by gwern on Thursday May 02, 2019 @02:13PM (#19810810)
> Where does Fisher point this out?

Probably in the Fisher excerpt.

Re:Everything Is Correlated by nonbel on Thursday May 02, 2019 @04:25PM (#19812450)
I looked but did not see it.