Statistics are routinely abused in nearly every field of science, alas.
The OP mentions findings about how forcing pre-registration on clicnicaltrials.gov seems to change results. Those are, of course, medical research studies. A discussion of those findings regarding pre-registration was on HN 10 days ago. https://news.ycombinator.com/item?id=10080680
I do not think it is conscious 'fraud' exactly. Sloppy statistics have just become routinized in standard scientific research in many fields. And yes, that sloppiness almost always errs in the direction of making it easier to show positive results, of course. But I don't think the researchers realize they are doing something wrong exactly, it's just totally routinized.
http://web.archive.org/web/20150115080542/http://wjh.harvard...
Is that what you meant?
(The archived link itself is dead, but he's listed as a PI here: http://www.wjh.harvard.edu/~scanlab/people.html)
The rejection of scientific method was what scared me.
But I don't know if statistics is the entire problem. It seems to me that a feature of good work in the "hard" sciences is that a problem will be approached from a bunch of different angles, so that the result doesn't really depend solely on statistics.
Synthetic chemists have to perform multiple orthogonal tests in order to announce a new compound, such as IR, NMR, X-Ray crystallography, and elemental analysis, on top of describing a mechanism for the compound's formation and performing reactions with the compound if possible.
Traditionally, to prove discovery of a new element, it wasn't good enough to see a blip on a mass spec. You had to do chemistry with it.
In my view a problem with data driven science is that you can do statistics without even knowing what you're actually studying. But if you don't know what you're studying, then you can't attack a problem from multiple angles because you don't even know if your tests are all measuring the same thing. You also have no tools to ask the hard questions about whether the results really make sense.
It is not uncommon to see published and "peer reviewed" work that makes sweeping claims off a tiny amount of self-reported data with nothing else to back it up. I have seen plenty of stuff that people on the "hard" sciences side of campuses would be embarrassed to even take responsibility for ... not to say they don't try to slip one by every now and again, but it seems systemic in the "soft" sciences.
Maybe physics, and to some extent, chemistry, are immune, I'm less familiar with them.
The issue is less stats, and more the pressure to publish, file drawer effects, poor career prospects for replication work relative to original research, fishing without multiple comparisons correction, etc. etc. etc. All of which combines to result in a lot of scientists looking the other way because the rules for stats and the rules for tenure aren't in alignment.
I saw all this first-hand when I was a grad student in cognitive neuroscience. Don't for a second believe this is limited to psychology. If anything, psychology is being more honest and grappling with these issues first.
It starts with the little things. I remember bringing up on HN the bullshit surveys most students do for their Master's thesis that are statistically worthless for the reasons said students don't understand. I've been told that hey, it's MSc, not a PhD, it doesn't have to be research-quality, etc. But hell, if people are willing to rely on bullshit methodologies for their Master's, when exactly do you think they'll learn the right way of doing things? Does the statistics god dispense additional levels of enlightenment when someone crosses the PhD threshold? I don't think so.
And honestly, it's not that much better in the hard sciences as well, on the undergraduate level. Maybe they won't let you get away with bullshit stats, but they'll happily let you get away with fabricating results and tweaking data to fit the model. Studying today is about only one thing - do everything you can to pass the exam. Research ethics? Never heard of it.
This can be made worse if you test several different hypothesis, only report successful experiments, and don't replicate them.
This doesn't surprise me at all. Kahneman wrote a book, "Thinking, Fast and Slow", in which he spends a long time talking about how most psychologists and social scientists do not understand the basics of statistics and therefore end up with flawed conclusions.
It seems like the majority of these highly intelligent people, despite spending a decade in training, were essentially statistically illiterate, even though statistics are absolutely central to their field. That's not a catastrophe in itself, you can always hit up the "real" statistician two floors down...but it seems like they didn't want to bother with that. Did they overestimate their ability or did they know they were serving up shitty research?
The papers that do replicate are tightly clustered a small number of researchers (Kahneman & Tversky)...I'm not sure how to interpret this. Did they get lucky and stumble upon the one area of psychological research that had a "high yield"? Did they implement a superior research methodology that nobody could/wanted to imitate?