Re:Curry spice turmeric boosts memory by nearly 30%, eases d...
by klmr on Thursday January 25, 2018 @06:27AM
(#16230175)
> but when a study size is small and you still find a difference, then it implies the effect size is large
That’s only true if it holds across multiple studies. Otherwise it might simply be random variability. By your argument, a study with n=4 that showed an effect would imply a huge effect size. No. It implies that the measured signal is highly variable, and that you got lucky on 1:3 odds. Check out regression towards the mean [1].
(Note that if you’ve got a good underlying model of the variability, n=4 can work. In fact, it’s routinely done in specific applications, such as transcriptomics. But even there it’s far from ideal, and it’s supported by a stringent error model.)
That’s only true if it holds across multiple studies. Otherwise it might simply be random variability. By your argument, a study with n=4 that showed an effect would imply a huge effect size. No. It implies that the measured signal is highly variable, and that you got lucky on 1:3 odds. Check out regression towards the mean [1].
(Note that if you’ve got a good underlying model of the variability, n=4 can work. In fact, it’s routinely done in specific applications, such as transcriptomics. But even there it’s far from ideal, and it’s supported by a stringent error model.)
[1] https://en.wikipedia.org/wiki/Regression_toward_the_mean