This isn’t my field, but I think this is par for the course unfortunately and a manifestation of a larger issue. Eg https://journals.plos.org/plosone/article/file?id=10.1371/jo...

I like to ask this whenever Dunning Kruger comes up; DK was a sample of Cornell undergrads, and the study was one tenth the size of the Stanford study. DK participants were volunteers of a psych class who got extra credit. Presumably they needed and could use extra credit, which may have excluded the A and the F students. It’s hard to imagine ways to start with more bias, or how we can possibly accept this sample as representative of humanity.

Social sciences are all a sham, now think about the trillions of dollars of government spending that are based on that same sham science as justification. People wonder why so many government programs fail, it's because they are built on a rotten foundation

https://en.wikipedia.org/wiki/Replication_crisis

>How can we conclude anything at all about the general population using a sample of Stanford kids?

it's a well known problem that is rarely brought up, WEIRD bias. Most social science research participants are college students being bribed with extra credit or gift cards

https://en.wikipedia.org/wiki/Psychology#WEIRD_bias

Good social science possible, but is difficult and (sometimes) expensive. If you can get the same personal outcome by doing something cheap and easy instead, of course that's what most people are going to do. Fixing that seems to be the Big Problem for most of science, for at least the last few decades (though, yes, particularly social science).

> People wonder why so many government programs fail

This, though, I'm not so sure about. Do government programs fail at a rate greater than those undertaken by other large organizations, like corporations or non-profits?

> Good social science possible, but is difficult and (sometimes) expensive

The problem is the system incentivizes bad social science.

That and people overstate and over interpret things.

There is nothing wrong with doing a study with faculty children. It is a great idea to get a bigger study done within a budget and timeframe. They write it up stating as much and their intelligent peers understand the caveats.

Unfortunately, the results are so exciting that it starts getting cited and taught by people who leave off the caveats. This is why it is important for someone to go back and follow up on the original experiment with new studies.