Re:Statisticians Find They Can Agree: It’s Time to Stop Misu...
by xtacy on Monday March 07, 2016 @03:37PM
(#11241427)
You are also normalising the difference by the variance, so the t-statistic has no units.
Re:Statisticians Find They Can Agree: It’s Time to Stop Misu...
by nonbel on Monday March 07, 2016 @04:22PM
(#11241799)
The claim the significance has nothing to do with the magnitude of a difference is just wrong though. It clearly does, this information is just merged with other information about the variance and sample size to get the p-value, which is compared to a threshold to get significance.
Re:Statisticians Find They Can Agree: It’s Time to Stop Misu...
by xtacy on Tuesday March 08, 2016 @12:47AM
(#11243740)
I think what the article meant to say was that, for the same number of samples, the p-value when you have delta=50 (the difference between groups), stdev=10 is the same as the p-value of delta=0.5, stdev=0.1.
Depending on the study, finding a delta whose p-value is significant does not necessarily mean that the size of the effect (i.e., delta) might be significant enough to be useful.
Here is the equation for a t-statistic (used for the common t-test):
d=mean(a)-mean(b)
s=sqrt(var(a) + var(b))/sqrt(n)
t=d/s
I see the magnitude of difference (d) right in the numerator. The t-value is then compared to the t-distribution to get the tail probability.