Re:Study finds 83% of software developers feel burnout
by anthonypasq on Tuesday July 13, 2021 @02:25PM
(#27824353)
have you taken intro statistics? all small sample sizes do is require your effect to be more pronounced to still be statistically significant.
Re:Study finds 83% of software developers feel burnout
by danenania on Tuesday July 13, 2021 @02:34PM
(#27824484)
Ah, then I guess that's why so few "statistically significant" studies end up being contradicted by further research...
Re:Study finds 83% of software developers feel burnout
by mactrey on Tuesday July 13, 2021 @02:49PM
(#27824653)
You make it sound like those studies were bad because the sample size was too small. In reality bad study design or having an unrepresentative sample are more common issues.
Re:Study finds 83% of software developers feel burnout
by danenania on Tuesday July 13, 2021 @02:59PM
(#27824803)
I think they go hand-in-hand. People naively apply intro-level statistics to messy real-world problems and then think they need an orders of magnitude smaller sample than they would need to even hint at the possibility of an interesting effect.
Re:Study finds 83% of software developers feel burnout
by majormajor on Tuesday July 13, 2021 @03:43PM
(#27825362)
Neither a bad experiment nor an unrepresentative sample will be helped by a larger sample size, though.
Re:Study finds 83% of software developers feel burnout
by danenania on Tuesday July 13, 2021 @04:01PM
(#27825586)
True, though no sample is ever truly representative along every dimension. Even when a lot of effort is put into making a sample as representative as possible, there will be variance in all the potential ways the sample can diverge from the population; this variance seems to be consistently underestimated.
Most studies are worthless due to small sample sizes and confounding variables, and most “conclusions” are based on cherry-picked outliers. The fact that it’s common to do this doesn’t make it responsible or correct.