News for hackers. Stuff that matters.
Re:Most published research results are false by arctangent on Friday February 11, 2011 @06:49PM (#2208701)
I always found statistics deeply disturbing when I was studying it at school.

You can establish a null hypothesis and then test at some confidence interval (say, 99%) and find that your null hypothesis still holds.

But, being somewhat ingenious, you may decide to lower the confidence interval (to, say, 98%) and find that your experimental evidence is now significant enough for you to reject your null hypothesis and accept your alternative hypothesis.

Lies, damned lies, and statistics, indeed.

Re:Most published research results are false by brent on Friday February 11, 2011 @08:08PM (#2208911)
This is why different communities accept p-values at certain levels (typically <0.05). You have to live with some amount of uncertainty when the data generating mechanism is random... that is unfortunately the nature of the beast.
Re:Most published research results are false by arctangent on Friday February 11, 2011 @10:37PM (#2209200)
I understand this. But let's say I ran some experiments and collected some data in an attempt to disprove theory X. What does it mean for me to say that at the 99% confidence level X is still true, but at the 98% confidence level it is not true? I just find it a bit spooky, is all.