News for hackers. Stuff that matters.
Re:Study finds 83% of software developers feel burnout by dragontamer on Tuesday July 13, 2021 @01:27PM (#27823642)
When we get into "total population" statistics, we kinda leave the realm of statistics lol.

There's 15 documented adverse events, but we don't really know how many adverse events there were in total. If we simply study those 15 adverse events, then your population study is only those 15 events (with a hell-of-a-lot of selection bias).

----------

But I'm mostly thinking about how one person I know thinks: "Did you hear about these 15 cases and even one of them died! Clearly J&J is unsafe to take!!!!"

For that kind of conclusion, its like they've focused their mind entirely on 15 cases, as if their "population of study" was 15. Its obviously bad logic, but maybe I wasn't very clear in my original post that I was referencing some bad logic there.

---------

But there's plenty of 50-person or 200-person studies that ARE properly done, and the science is solid. Especially when we're looking at say Phase 1 / Phase 2 style trials of safety and/or efficacy (the earlier stage trials before a Phase 3 trial is conducted).

Its not a big enough sample to be "certain" of the results, but its a big enough sample to "provide evidence", possibly in conjuction with other studies.

----------

In many cases: having 5 or 6 studies with 50-random people each and performing a meta-study across them (especially if all 5 of those studies were of high quality sampling) would lead to a more robust conclusion than just one 1000-person study done slightly incorrectly (ex: the 1000-person "study" is a voluntary telephone survey or some other low-quality study)

Re:Study finds 83% of software developers feel burnout by NotSammyHagar on Tuesday July 13, 2021 @02:15PM (#27824231)
Sorry but I disagree. The key point is that over a very large population, millions, only 15 people got it. It's just wrong to say it was computed over only 15 people.

This study referenced up above had the ~285 or whatever people who answered the survey, not that much info. ~285/350 is not that much data.

Anecdotally and based on my own experience I'd agree that the burnout is high, probably over 50%.

Re:Study finds 83% of software developers feel burnout by brewdad on Tuesday July 13, 2021 @03:26PM (#27825130)
Would you agree that we can conclude that the incidence of burnout is high amongst software developers? The "real" level may be 83% or 93% or 43% but it seems fair to conclude that it is indeed some larger than desirable number that merits a larger and more robust study.

That is the value of a study of this size. The ability to quickly determine if there is any "there" there. The headline is sensationalistic but such is the nature of publish or perish science today.

Re:Study finds 83% of software developers feel burnout by spywaregorilla on Tuesday July 13, 2021 @02:19PM (#27824279)
> When we get into "total population" statistics, we kinda leave the realm of statistics lol.

Not even close to accurate.

Re:Study finds 83% of software developers feel burnout by dragontamer on Tuesday July 13, 2021 @02:41PM (#27824556)
When there's an election or census over some particular figure, there's no "+/-" or "standard deviation".

If the election count is say, 48% for A and 50% for B and 2% other, that's that. You don't have error bars, you don't have student-t tests, you don't have random varaibles, you don't have "confidence intervals". You assume you have the ground truth. You have a precise number assumed to be without error.

Its actually a fundamentally different kind of math than what is typically taught in statistic classes (standard error vs standard deviation. R-values, random variables, etc. etc.)

When it comes to "total population" style information gathering, the questions of reliability are about whether-or-not you in fact, grabbed the total population... and nothing really "mathematical" (such as what are my 95% confidence interval?)

---------

When people say 600,000 died of COVID19 in the USA, there's no error bar, standard deviation, or random variable on that number. It is what it is: the ground truth according to a particular source.

And when you have a 2nd source of information with slightly different numbers (ex: State databases vs US level databases), you have a non-mathematical problem for how to resolve the apparent contradiction.

There's no "metastudy" over other surveys. There's no "resample again", there's no reproducibility. These statistical concepts and tests simply do not apply. If you do in fact decide to conduct a 2nd census, you will find that the numbers will differ in practice (ex: recounting the same set of ballots a 2nd time). But there's no underlying mathematical model for why this happens, as is the case in typical population statistics (student t-test assumes a standard error over a standard deviation of a hypothetical sample, etc. etc.)

Re:Study finds 83% of software developers feel burnout by spywaregorilla on Tuesday July 13, 2021 @03:08PM (#27824897)
Elections are not statistical tests. Number of people dead are not statistical tests. You said "When we get into "total population" statistics, we kinda leave the realm of statistics lol." That is false. There are many questions that statistics is best suited to answer even if an entire population has been surveyed. Often the term population is poorly defined, and is much wider than the simple base case.

In the case of J&J blood clots, there's billions of additional people who could take the J&J vaccine. 15/21 million people who took he J&J vaccine got blood clots. But if you want to know what percentage of "people who take the J&J vaccine get blood clots" that's not the same thing. The population includes an infinite number of people.