News for hackers. Stuff that matters.
Re:Most commonly used statistical tests and implementation in R by GFK_of_xmaspast on Friday January 01, 2016 @09:27AM (#10821950)
> could as much as 30% chance you are wrong about the actual difference being there depending on the true probability of the initial hypothesis.

I'm having trouble parsing this, are you talking about the power of the test?

Re:Most commonly used statistical tests and implementation in R by capnrefsmmat on Friday January 01, 2016 @10:56AM (#10822159)
No, the base rate. If you're doing a significance test to detect a rare disease that only occurs in a tiny fraction of people, for example, the majority of statistically significant results you obtain will be false, despite having p < 0.05.

This is a common problem in many fields, and you can use false discovery rate control methods to account for it: http://www.statisticsdonewrong.com/p-value.html