News for hackers. Stuff that matters.
Re:Could It Be? Spooky Experiments That 'See' The Future by Eliezer on Tuesday January 04, 2011 @11:17PM (#2069862)
I repeat, you do not need 1000 times as many subjects to get results that are 1000 times as significant! If 40 subjects gets you results with p < 0.05, then 100 subjects should get you results with p < 0.001. Doing half as many experiments and having nearly all the published results being real effects, instead of most of them failing to replicate when tested, sounds like a great tradeoff to me.

And I suspect the ultimate reason it's not done this way... is that scientists in certain fields would publish a lot fewer papers, not slightly fewer but a lot fewer, if all the effects they were studying had to be real.

Re:Could It Be? Spooky Experiments That 'See' The Future by saurik on Tuesday January 04, 2011 @11:36PM (#2069919)
"1000+", not "1000x". Also, I'm assuming bigfudge was talking about p < 0.0001, given the comparison made to physicists.
Re:Could It Be? Spooky Experiments That 'See' The Future by bigfudge on Wednesday January 05, 2011 @09:35AM (#2071050)
Yes - thanks. The current norm for a 'suitably powered' trial of a psychotherapy is about 300. We've just got a trial funded for that number (admittedly in a challenging patient population) which will cost about £2.5m in research and treatment costs. We would love to run 1000 patients and start looking at therapist-client interactions, individual differences in treatment suitability but that's out of the question.