News for hackers. Stuff that matters.
Re:A new form of brain stimulation relieved severe depressio... by beerandt on Tuesday April 07, 2020 @11:29AM (#22804096)
Isn't publishing anything with N<30 that presumes a random distribution straight-up wrong?

(I don't know if that's presumed here or not, but without greater detail, it's something most people [who have a basic understanding] will assume.)

Re:A new form of brain stimulation relieved severe depressio... by renewiltord on Tuesday April 07, 2020 @11:34AM (#22804151)
You have 29 people in a room. The people are healthy and well. You give them a pill containing a chemical you want to test. They all die seconds after consuming the pill. The autopsy reveals they did not choke. What is the N you choose before you conclude that chemical is toxic in that dose at some p?
Re:A new form of brain stimulation relieved severe depressio... by beerandt on Tuesday April 07, 2020 @12:00PM (#22804398)
That's not what I said- you can still make claims against the data.

But when people are looking at N=x, most don't realize that N<30 is usually in a completely different category than N>=30. You almost shouldn't look at it as "N".

Novice "statisticians" are just (very) limited in the tools/formulas/methods available to use, because so many (entry level/basic/standard) statistical methods are only valid for normal distributions (or other constraints on data, that people typically don't check for).

Without details stated up-front, most people will (wrongly) assume that a normal distribution automatically applies, even if they don't know what that means. Similar to situations where people ask for the average/mean when they should be asking for the median.

The data isn't useless, but needs strong context and disclaimers.

Re:A new form of brain stimulation relieved severe depressio... by IAmEveryone on Tuesday April 07, 2020 @01:06PM (#22805126)
I have no idea what you’re getting at here. The parent pointed out, accurately, that for large effect sizes, N=30 is plenty.

I’m also fairly certain you don’t understand what a “random distribution” is yourself. That’s because there’s no such thing. There’s a normal distribution, binomial distribution, Poisson distribution etc, and they all involve randomness. But there’s no “random distribution”.

So maybe you shouldn’t be quite so quick to judge “novice statisticians”, and what those imaginary people you mention a total of five times may or may not “wrongly assume”, or “don’t know”, or “are limited in”, or “don’t check for”.

Re:A new form of brain stimulation relieved severe depressio... by beerandt on Tuesday April 07, 2020 @02:12PM (#22805821)
You're right, I meant normal, not random. (Fixed)

And I didn't mean to imply that I'm any better than a novice. I just know enough to know to always check that assumptions for any given test needs to be valid. (It's been a few years since I've had to use any of this in a professional capacity.)

But is 30 plenty? Or is it sufficient?

As I remember, normal distribution usually requires a sample size of N=>30. (As in 30 is the minimum.)

Re:A new form of brain stimulation relieved severe depressio... by markroseman on Tuesday April 07, 2020 @01:30PM (#22805398)
On the other hand, there are so many opportunities for experimental design limiting the applicability of results in mental health experiments, that sample size and distribution doesn't even begin to touch on it. Bottom line: getting a meaningful result that is widely applicable is a horrendously complex undertaking.
Re:A new form of brain stimulation relieved severe depressio... by renewiltord on Tuesday April 07, 2020 @05:00PM (#22807416)
Indulge me with the scenario. Do you think that pill is safe or unsafe or do you think we need to give one more guy the pill to test?

If you think it depends on something, go ahead and say so and I'll constrain the thought experiment.

IAmEveryone (user w/ sibling post) is correct in that I'm referring to the fact that effect size affects statistical power.

Re:A new form of brain stimulation relieved severe depressio... by afpx on Tuesday April 07, 2020 @12:11PM (#22804507)
A lot of times they will do small experiments to gain a sense for the data and see if it's worth doing larger experiments. Not reliable results but can still infer things.
Re:A new form of brain stimulation relieved severe depressio... by beerandt on Tuesday April 07, 2020 @02:41PM (#22806109)
Yeah, I didn't mean to imply that the knowledge isn't or couldn't be useful.

In hindsight, it was too early in the day for me to try and speak statistics.

Re:A new form of brain stimulation relieved severe depressio... by ramblenode on Tuesday April 07, 2020 @07:08PM (#22808443)
(I did not downvote)

I have also observed people suggest a statistically special N, but this is a misconception. There is no special N. There is only a relationship between increasing N and a monotonic increase in the confidence in your result. Most scientific standards specify a "confidence" of at least 95%, but if you are willing to accept more uncertainty--perhaps because there is a large upside to a positive result--then you can make due with a smaller sample.