News for hackers. Stuff that matters.
Re:Nootropics – The Facts About “Smart Drugs” by Permit on Thursday September 13, 2012 @11:18AM (#4516570)
>Warning Sign D3: Too Few Subjects >The author advertising smart drugs doesn't describe any careful studies, so of course he doesn't mention the number of subjects in any studies. Most so-called studies of smart drugs have been far too small to have any statistical power.

Neither yourself, nor Peter Norvig suggested an appropriate number of subjects for tests like these. So I'm curious, what would be an appropriate number of subjects?

Re:Nootropics – The Facts About “Smart Drugs” by lars on Thursday September 13, 2012 @11:52AM (#4516812)
You test some statistical hypothesis, and you end with some probability that the hypethesis is true. Increasing the number of people increases the number of samples. Loosely speaking, increasing the number of samples will reduce the variance on the variable you are measuring. When the variance is reduced, the confidence in the result is increased.

Say you are measuring IQ before and after taking omega 3. Say your hypthesis is "omega 3 immedieately increases IQ". Assume this is true, even if it probably isn't. After testing this on 20 people you might be able to say "the hypthesis appears to be true, but there is a 46% percent chance these results are due to chance". However after testing 1000 people, you might be able to say "the hypthesis appears to be true, and there is a 0.5% percent chance these results are due to chance". Then you are 99.5% confident in your result. Standards for confidence varies from field to field, but typically 99% confidence or more is called a significant result. Given some assumptions, you can also calculate the number of samples needed to generate a result with confidence >X% with probability Y. According to Daniel Kahneman in Thinking Fast and Slow, most researchers don't take the time to do this, and are consistently surprised when their experiments turn out to not give significant results.

Re:Nootropics – The Facts About “Smart Drugs” by Evbn on Thursday September 13, 2012 @10:53PM (#4519793)
This post, like so many others, states the p-value concept backwards. Statistics give you the chance of a false positive observation among a universe of negative realities, not the chance of a false reality in a universe of positive observations.
Re:Nootropics – The Facts About “Smart Drugs” by stephengillie on Thursday September 13, 2012 @01:15PM (#4517355)
Re:Nootropics – The Facts About “Smart Drugs” by pnathan on Thursday September 13, 2012 @12:51PM (#4517197)
> So I'm curious, what would be an appropriate number of subjects?

This depends on the experiment. Some of the things to account for are the number of variables, the confidence you are looking for and the amount of interdependence. Consult a statistician, particularly one expert in experimental trials, for an evaluation.

Re:Nootropics – The Facts About “Smart Drugs” by tel on Thursday September 13, 2012 @01:54PM (#4517592)
Rule of thumb, the larger the effect size, the fewer the number of participants. The larger the random variation across people the greater the number of participants.

There are also exact formulas for simple tests.