This is why it’s better to have tiers of confidence for actual policy decisions and not just confusion matrix results frankly.
Like if you run this test and you’re in a cohort that is 99.999% likely to be cancer, that’s useful info. It’s not a problem, per se, if the test instead comes back with an answer of mixed certainty. We just need to be comfortable with getting neither a positive nor a negative prediction; nor expecting it to cover all cases.
The policy driving thresholds on tests need to consider the weights of each possible outcome to determine what to do.
A normal human fallacy is to focus on the thing you understand, that is easy to understand (e.g., easy to quantify), and overlook the difficult issues that are far more important. There is much more, of much more importance going on in most of these studies, and in their statistics and validity, than the sample size.
I first took stats in high school and we read a bunch of valid small sample size studies so I don’t know where people are failing to get educated.
There were so many factors going into whether a study was good.
Famous example: Lady tasting Tea [1]. N=1 or N=8 depending on how you look at it. Still significant.
This is the one that proves that it's possible to tell whether you added the milk first or second.
> Now there are 8 billion people, so it's worth 16 times less.
That is not how statistics works, at all.
> In fact, this is why I always trust science in small towns more. In Colma, for instance, you sample 500 and you have covered 33% of the people.
These are the sentences that made me think this had to be satire (presumably you want "science" to apply to places outside of Colma...), but in all honesty these days it's really hard to tell.
To all who criticize and ridicule someone who would like to have more samples, why do you think 21 is such a perfect number in this case? Wouldn't 15 be enough, if statistics and all applies? 10? 1? Would 30 be too much?
This is the point that you are fundamentally misunderstanding. Nobody is making the argument that "21 is such a perfect number in this case". The rest of your sentences ("Wouldn't 15 be enough, if statistics and all applies? 10? 1? Would 30 be too much?") seem to point to a belief that people are pulling numbers based on a finger in the wind.
The whole point of (a lot of) statistical testing is that it allows you to come up with a specific number that determines how likely a particular result is due to chance. That is what the p < .05 "standard" is about - it's a determination that the results have less than a 5% likelihood to be due to random chance (though that 5% number used as "significant" is just basically pulled out of thin air, and p-hacking is another topic...) That is, I and the comment I replied to aren't making the argument that 21 "is such a perfect number". We're making the argument that even with a small sample size it's possible to determine the relative error bars, with precision, using statistical methods, not just pulling a number based on feels. Yes, larger sample sizes reduce the size of those error bars. But often not in ways that are "intuitive". You have to do the math.
None of what I wrote above is meant to imply that statistical tests can't be misused, or that they often require assumptions about the underlying population distribution that may not be correct.
Effect size is going to be important when talking about clinical significance, not just statistic significance, too.
I also want to point out that we have no stats on sensitivity, specificity, or diagnostic odds ratio, which are all clinically relevant to physicians deciding when to test and how to interpret test results.
The good news is that it's a non-invasive, low cost test which plays a factor in clinical decision-making.
One thing I'd note, though, is that the paper's title is "High sensitivity saliva-based biosensor in detection of breast cancer biomarkers: HER2 and CA15-3". My understanding is that sensitivity was never really the problem with breast cancer detection - it's specificity that is the real challenge with all types of broadly-deployed medical screening tools.