It is to the merit of psychology. Other branches of social sciences don't dare to propose such a thing. They know that their replication numbers are going to be worse than psychology's.

Most social sciences have numbers that are far better than psychology's.

https://royalsocietypublishing.org/doi/10.1098/rsos.200566

For fuck's sake, marketing and management has better numbers than psychology. It is to their merit they're admitting they have a problem, but it is 100% a problem of the field of study.

You misrepresent the linked article which doesn't report replication rates but forecasted replication rates from participants "recruited through a number of mailing lists, Twitter and blog posts".

They used a prediction market, which at the time the article was published was still to resolve. This is standard practice and methodologically sound. They didn't go around asking people on Twitter.

Just skimmed it quickly, but from [their paper](https://royalsocietypublishing.org/doi/10.1098/rsos.200566 ):

> Participants were recruited through a number of mailing lists, Twitter and blog posts. [...] For all analyses, we use survey responses; for analysis (iii), we additionally use responses to a demographic survey that included a question on academic interests; and for analysis (iv), we additionally use the prediction market data.

So it sounds like they only used prediction-market data "for analysis (iv)", which they describe as:

> [...] (iv) whether survey-based aggregated forecasts and market-based forecasts are correlated.

---

The last line of their paper:

> If our forecasts hold up, it will be interesting to investigate if specific factors (such as different methodologies and policies) can be identified that influence replication rates.

So, it doesn't sound like they're claiming that their forecasts should be assumed reliable.

No, it isn't. A prediction market is not proof of anything.

I didn't say that the methodology was not standard practice or methodologically sound. I said you misrepresented the study, which you did. You said "Most social sciences have numbers that are far better than psychology's." in the context of a conversation about *replication numbers*. You did not say "in a prediction market, participants ("recruited through a number of mailing lists, Twitter and blog posts") have lower replication expectation rates for Psychology than other social sciences" -- if you had then, well, you wouldn't have misrepresented the study.

Any meta-study trying to get replication numbers would likely use a combination of methods, including prediction markets, with techniques like having a small fraction of the contracts resolve.

So no, implying this is a Twitter poll is a far worse misrepresentation than even the most uncharitable interpretation of what I said.

> Any meta-study trying to get replication numbers would likely use a combination of methods, including prediction markets, with techniques like having a small fraction of the contracts resolve."

this is non-nonsensical obfuscation.

poor attempt at moving the goal posts -- for the record I did not imply, I DIRECTLY QUOTED from the article.

Not in the field, but this recent podcast covers it:

https://podcasts.apple.com/gb/podcast/two-psychologists-four...

Harkening back to episode 73, Alexa and Yoel discuss recent evidence of fraud documented in the Data Colada blog post "Clusterfake." The post is the first in a series of four, which will collectively detail evidence of fraud in four papers co-authored by Harvard Business School Professor Francesca Gino. […] Finally, they consider what this means for a field still struggling to build a more trustworthy foundation.

What's the other epicentre?

They've had a bunch of trouble in medicine-adjacent fields. Nutrition and sports science has had an especially tough time, but it's been cropping up all over the field.

I do think most of the problem is rooted in the fact that it's difficult to do this type of science, especially without serious ethical braches. There are so many confounding variables, and self-reported observational data just isn't a very good tool. To truly eliminate confounders, you basically need to lock someone up for decades and control their entire lifestyle.