I think that's true for low-tier/low-impact CS papers, but the difference is that literally nobody cares about those papers. The high-tier stuff is easily verifiable (like Tensorflow or whatever) and nobody is writing articles about 5% improvements against a benchmark in some obscure niche scheduling and planning domain.
Outside academic CS people are more scientific about experimental results because it has concrete implications on revenue or spend... but they aren't getting the results from conference papers.
Outside the high-profile cases it seems accepted norm that papers perform far worse when scored against somebody else's benchmark. The real measure of quality is how big the gap is.
Adding to complication in the psych field is the widely observed phenomenon that it draws students with psych problems. Like those who would be willing to fabricate data, for example.
But hey, at least it isn't sociology.
There are resilient exceptions in academia in general, but soft science fields have led the way in the decay of academic standards.
There are dozens of well known fields that are built on epistemological quicksand yet which point-blank refuse to admit to or talk about their problems. Because they have a culture of deny deny deny, even when the evidence is overwhelming, they actually get less attention because why bother doing a nice writeup of fraud if you know the result will just be stonewalling? It's better to focus on fields where there might be some actual response, a chance of improvement, no matter how minor.
Some results do replicate. Big Five personality traits, general classification of mental illness, mainstream IQ results, and human perception and performance. All psychology is not false and there are interesting things to learn from the field.
But the severity and frequency of fraud and non replication is worst in psychology. So much so that half of what you learned in Psych 101 ten years ago does not replicate.
The big challenge for the field is that humans know a lot of psychology. We can track our own thoughts and we are constantly interacting with people and trying to understand their psychology. A biologist who studies ants intensely for 5 years is maybe only one of 100 people have ever watched them that carefully. They'll find lots of new stuff.
Psychology doesn't have powerful techniques than that for determining new truths about humans. It's still mostly give a questionnaire or put people in weird situations.
So there are a lot of researchers hunting around for original ideas that are undiscoverable with our current tech. Some of them are bound to give into the temptation to just make up an interesting result with manufactured data. With p hacking they might not even be committing fraud, they are desperate for a positive result, when the get one they stop looking and publish.
So I would still guess in 2023 that about 80% of "new" discoveries published in journals won't replicate. That rises to 95% with journal publications that are published as university pr and reported in the media.
For coverage of what to remember to unlearn Rolf Degen is great: https://twitter.com/DegenRolf
Popular examples that do not replicate (mostly from Rolf):
Repressed childhood memories Social media harms Search bubbles and echo chambers Women prefer masculine men when fertile Priming (you see a fight in the hallway and then don't cooperate later) Power posing (puff out your chest to feel more confident) Watching eyes make people more honest (in Kahneman and Malcolm Gladwell)
Notice that a lot of these are interesting, and you want them to be true. That's not enough to make it so.
As a bonus, an interesting interview with Daniel Kahneman: https://www.edge.org/adversarial-collaboration-daniel-kahnem...
My experience in CS is that the replicability of experimental results is embarrassingly bad, but this isn't making headlines in the same way so people don't consider it to be a problem.
This is data fraud in psych. Data fraud has also happened in plenty of other fields. When it occurs in these fields it is seen as a one-off. When it occurs in psych, it is because the entire field is useless garbage. That's not the lesson to take away here.