News for hackers. Stuff that matters.
Re:Most commonly used statistical tests and implementation in R by ekianjo on Friday January 01, 2016 @03:20AM (#10821406)
> If the p-Value is less than significance level (ideally 0.05),

Erm, no. P=0.05 is borderline meaningless, there could as much as 30% chance you are wrong about the actual difference being there depending on the true probability of the initial hypothesis.

P-values should be used with strong caution.

Re:Most commonly used statistical tests and implementation in R by snydly on Friday January 01, 2016 @03:52AM (#10821442)
> P-values should be used with strong caution.

FiveThirtyEight (and Scientific American, and others) did some pretty interesting articles about this recently if you haven't seen it:

http://fivethirtyeight.com/features/science-isnt-broken/

Just from personal experience, the use of p-values is really broken in biology/chemistry. The things I've heard principal investigators say...

Re:Most commonly used statistical tests and implementation in R by cetacea on Friday January 01, 2016 @01:32PM (#10822733)
Even better, p-values should not be used at all. If I have data in hand, I want to use it to find out the probability that my hypothesis is true. But p-value analysis requires me to instead ask a different question that I don't really care about, involving whether my data are consistent with the null hypothesis.

Everything is just so much more sensible if you allow yourself to assign probabilities to hypotheses, rather than assuming a hypothesis from the outset and computing opaque statistics relating to your data.

Re:Most commonly used statistical tests and implementation in R by healer on Friday January 01, 2016 @04:12PM (#10823476)
There is in fact a probability attached to p-values. A p-value of 0.05 for instance means your conclusions will be wrong 5 out of 100 times. You can reduce the p-value to e.g. 0.001 or any other value you want.
Re:Most commonly used statistical tests and implementation in R by cwyers on Friday January 01, 2016 @05:13PM (#10823725)
No, it means that the probability of seeing an effect of that magnitude on a dataset of that size when the null hypothesis is true will happen due to random chance 5 out of 100 times. It says NOTHING about your hypothesis, it is entirely a statement about the null hypothesis.
Re:Most commonly used statistical tests and implementation in R by GFK_of_xmaspast on Friday January 01, 2016 @09:27AM (#10821950)
> could as much as 30% chance you are wrong about the actual difference being there depending on the true probability of the initial hypothesis.

I'm having trouble parsing this, are you talking about the power of the test?

Re:Most commonly used statistical tests and implementation in R by capnrefsmmat on Friday January 01, 2016 @10:56AM (#10822159)
No, the base rate. If you're doing a significance test to detect a rare disease that only occurs in a tiny fraction of people, for example, the majority of statistically significant results you obtain will be false, despite having p < 0.05.

This is a common problem in many fields, and you can use false discovery rate control methods to account for it: http://www.statisticsdonewrong.com/p-value.html