Re:The brains of lonely people respond more negatively to so...
by roymurdock on Thursday September 24, 2015 @11:14AM
(#10272053)
Got it, thanks for the explanation. I generally trust that the authors of such studies know what they're doing when they design an experiment with a certain n value in mind. Agreed that the insufficient n argument does crop up a lot around HN.
Re:The brains of lonely people respond more negatively to so...
by sliverstorm on Thursday September 24, 2015 @03:49PM
(#10274167)
Also important, I recall a note from one of my statistics classes to the effect of, strong sample sizes are often counter-intuitively small.
I want to say the example was something like polling in a political race, and the math worked out that you could get reliable results polling only a few tens of people so long as you had random sampling.
Re:The brains of lonely people respond more negatively to so...
by roymurdock on Thursday September 24, 2015 @04:25PM
(#10274436)
One of my professors had a rule of thumb of discounting anything below n=25, but you absolutely need to ensure random sampling as its one of the fundamental assumptions of regression - making sure the probability of being part of the sample is uniform across the entire population.
I don't disagree with this at all. You always want to increase your sample size if possible. But as you also say, the magnitude of the effect you're observing is also important; if it's very strong, then it can be revealed by a small sample size. An absurd example could be "cyanide capsules kill (n=2)". Obviously, the smaller the sample size, the harder it is to draw reliable conclusions, but I wouldn't think n=19 is necessarily too small. I think many of phase I clinical trials aren't much more than that.
More importantly, mentioning a "small" sample size makes a nice soundbite, and is often used to discredit studies. These studies might be good or they might not be, but talking about sample size can be used as a "thought-terminating cliche". Perhaps a person is looking for evidence that reinforces their own beliefs (we all do it), and they read a criticism of a study, it mentions "small sample sizes", and they can walk away happy in a belief the study was flawed. But all that's really happened is that it's prevented a proper discussion of the study taking place. I'm not sure how much of a problem this is, but I've seen it happen frequently.