I hate to rain on your parade, but sample size has everything to do with whether a result is statistically significant.
http://sph.bu.edu/otlt/MPH-Modules/BS/BS704_Power/BS704_Powe...
If you want to test whether penicillin can cure a staph infection, you can get statistically significant results with a handful of tests.
>> I hate to rain on your parade, but sample size has everything to do with whether a result is statistically significant.
> No, almost any sample size can be sufficient, as long as the effect is big enough.
Your sentence says "no", but it agrees -- sample size has everything to do with determining statistical significance. The ratio of sample size to population is critical to deciding whether a result is significant: http://classroom.synonym.com/select-statistically-significan...
> Though in psychology, larger samples are often needed, because there's generally smaller effects.
Yes, but many of those kinds of result are insignificant and instantly forgotten regardless of the circumstances, because psychologists generally aren't testing a falsifiable theory, only measuring an "interesting" effect, like whether leaning to the left makes the Eiffel Tower look shorter (the 2012 Ig Nobel Prize winner):
Title: "Leaning to the Left Makes the Eiffel Tower Seem Smaller -- Posture-Modulated Estimation"
Link: http://pss.sagepub.com/content/early/2011/11/23/095679761142...
Ig Nobel Prize announcement: http://www.improbable.com/ig/2012/
I don't think anyone on hn won't know this. Everyone is just quibbling over the wording.
Your new point is very good, and I'd expect someone has used the correlation != causation argument too.
The study makes no mention of controlling for degree programs or a variety of other factors which would affect cognitive development, combined with the incredibly small sample size, It is my opinion that any conclusions from this research need to be taken with a pinch of salt.
A larger sample size means that a smaller average increase is required to show that it is a statistically significant deviation than a smaller sample.