> they recruited 2,400 people to join four custom-made social networks
Also there was a similarly sized study a few weeks ago, and basically all participants were students or faculty at 2 different universities.
If you are going to make a claim like this, you need to have a very large and very significant measurable effect, and the smaller your sample size the larger the effect needs to be to have any confidence of the correctness of the study.
Again, to be on HN the paper itself, and the source data, should be freely available. By allowing them to be on HN as they are we are essentially allowing unchecked press releases that always over state the validity of claims.
For a population size of 350,000,000 people the required sample size for a 95% confidence level within a 2% margin of error is 2,401 people. Assuming these 2,401 people were selected properly then that is just about enough, even for a claim like this.
Also, if you ask me, 95% confidence with 2% error margin is pretty low bar.
welcome to psychology studies
Especially these “doing X causes [major behavioral change]” papers where the sample size is super important, and they also tend to have very low reproducibility rates?