When I was in grad school I was doing graph theoretic analysis of human metabolic networks. I had some really interesting and promising preliminary results about the structure of various metabolic pathways and had been invited to present my unpublished work in a multi-departmental lecture. I was making some new visualizations for my talk and spotted something odd, which led me to a bug in my source code that had reversed the direction of several steps in many of the pathways. I was mortified . . . I ran some preliminary tests on the corrected code and saw that many of my results were completely in error. I wouldn't know for sure until I could get several days of cluster time, but I had proof that my last year of work was completely bogus.
I stayed up all night confirming the new code and the extent of my invalidated results and brought this to my advisor the morning before the lecture. He told me to shut up and present it anyways. Being a young idealist I argued with him about the nature of science and the search for truth . . . I refused to go along with it and present something I knew was false. He made one of the research scientists present on their topic at the last second so our lab wouldn't "lose face" and I was pretty much in the dog house forever after.
This is how Science should be done.
A major conceptual difficulty was the somewhat nebulous definition of a metabolic pathway, beyond what reactions were drawn together in a textbook diagram. Also somewhat thorny was the concept of building a network with any sort of transitive property with a notion of mass conservation built in. I wrote about 30k lines of terrible Perl code exploring this stuff, if I had independent backing I would definitely dust it off and finish the project but it's pretty far in the rearview mirror at this point.
Being naive, I just assumed they'd be retracting them, but my labmates explained there was no way our PI would let that happen. It was never boiled up to the PI, so he never even knew. A year later (due to this as well as other issues), I quit the PhD and joined a company doing something "real".
Having journals/conferences with things like double-blind peer-review processes in place to me is probably the only way to do this. How else would you filter all the noise?
We don't have this in industry and it's a bit sad. I mean we do in the form of some blogs, conferences, etc. But in no way it compares to the system academia has. Have you ever tried publishing to an academic conference compared to an "industry conference"? All you need for most industry-conferences is an abstract, an idea, not even evidence of new sorts; actually you need reputation but that's a different issue.
There are flaws in every system for sure and academia has their hands full of them especially since they have to deal with quite important things like medicine.
But I find it funny that people complain about academia, and yet there's no other system that comes close to producing empirical truths.
As for designing a better system that's more robust to these flaws, it is a hard problem. There are structural issues when trying to apply traditional scientific virtues to the real world (how do you reasonably ensure reproducibility with a ridiculously expensive experiment, or one spannig decades, or one involving cohorts of tens of thousands of people? How do you ensure peer review is honest and comprehensive when there are exactly three labs specialized in a subject, and they're all competitors? Or even collaborators?), even if you assumed that editorial bias (toward big names, against negative results, toward trendy subjects, etc.), dishonesty and nepotism didn't exist. Plus there's a wide system of power-wielding stakeholders with huge incentives to maintain the status quo: established big shots, publishers (cough cough Elsevier cough cough), and so on.
So yeah, things are broken but we're all trudging along trying to enjoy what we do and make do with what we have. Who knows, things may even get fixed one day.
In my view it should be a judgment call whether a particular result needs to be replicated. I'm aware of the so called replication crisis. But adding more arbitrary rules will simply invite people to figure out how to game those rules, if they were gaming the original rules (deliberately or unwittingly).
Imagine, I will report today that I'm able to determine temperature of the physical vacuum (e.g. using double slit experiment), and then able to drop it, i.e. cool vacuum below absolute zero. Will you trust me? Of course, not. Then when you will trust me?
Interesting fact: the majority of the impact factor values of big-name journals (Nature, Science, Cell and co.) comes from a few select high-impact papers. Most of the rest goes relatively uncited. The reason is, most of these papers are not astounding but simply make it through the editorial system because the big shots in charge of the labs know the editors. Not that these papers themselves are devoid of merit (though I know my fair share of Nat/Sci/Cell papers that happen to be 80-100% bunk), but they wouldn't have made it to such highly selective journals otherwise. These journals also happen to have an usually high retraction rate. I have friends working in the aforementioned big-shot labs that are jaded to tte point that they just dismiss most thing coming out of Science.
And that's just my area and that of people I know. Things go very awry in medicine or anything related to cancer where the pressure is so high, the field so competitive, the scientific questions so complicated, the experiments so hard to reproduce and the incentives to publish so exacerbated (note that medical journals generally have the highest impact factors). And I can't even begin to imagine what goes on in psychology.
Plus, the pay is lousy.