News for hackers. Stuff that matters.
Re:The blogger who spotted problems in Dana-Farber cancer pa... by ethanbond on Friday January 26, 2024 @10:31PM (#39152223)
Well yeah when you add stuff I didn’t say, it does sound ridiculous.

I didn’t say they would reject it based on the result and presuming an error. They would find and reject the error (or on the basis of a million far more superficial issues which are actual good targets for reforming scientific publishing).

Re:The blogger who spotted problems in Dana-Farber cancer pa... by andrewflnr on Friday January 26, 2024 @11:44PM (#39152658)
What did I add that's meaningfully different from what you said, i.e. in terms of outcome?

Ed: ok, I think I get it, you mean they'll somehow identify a specific method error, and if they can't then they'll... Just accept it? I simply don't believe that. I think the journal both should and actually will reject papers that violate the second law of thermodynamics without finding a specific methodological flaw.

Re:The blogger who spotted problems in Dana-Farber cancer pa... by superb_dev on Saturday January 27, 2024 @01:00AM (#39153023)
If the paper is completely methodologically sound and the only problem is that it violates the second law of thermodynamics, then we need to take another look at the second law of thermodynamics
Re:The blogger who spotted problems in Dana-Farber cancer pa... by andrewflnr on Saturday January 27, 2024 @01:37AM (#39153208)
You're assuming that (a) reviewers not finding any methodology flaws in the paper means there are none, and (b) no flaws in the paper means no flaws in the work. Neither of those are good assumptions. And that's just the generically applicable arguments.

For the Second Law, Arthur Eddington had this to say:

> ...If someone points out to you that your pet theory of the universe is in disagreement with Maxwell's equations - then so much the worse for Maxwell's equations... But if your theory is found to be against the Second Law of Thermodynamics I can give you no hope; there is nothing for it but to collapse in deepest humiliation.

https://www.goodreads.com/quotes/947685-the-law-that-entropy...

Re:The blogger who spotted problems in Dana-Farber cancer pa... by ethanbond on Saturday January 27, 2024 @08:29AM (#39155375)
> You're assuming that (a) reviewers not finding any methodology flaws in the paper means there are none, and (b) no flaws in the paper means no flaws in the work

No, you are assuming that with your flawed understanding of what a journal is and what peer review does. Like this isn’t a matter of opinion. Peer review in fact and by design is not a stamp of veracity. The reason is exactly the conundrum you’ve backed into.

If someone produces apparently high-quality science that challenges a well-regarded theory, it is only under your model that paper cannot be published. Under our model (the one currently in use today), the reviewers are expected to try to find holes in the methodology and, if they can’t find them, publish the paper anyway even if they disbelieve the conclusion. That way the broader scientific community can attempt to blow holes in the paper, and they frequently find things the authors and the peer reviewers missed. That is not a shot against the publisher: that is how science must work because of exactly the dynamic you’ve identified.

Re:The blogger who spotted problems in Dana-Farber cancer pa... by andrewflnr on Saturday January 27, 2024 @12:40PM (#39157773)
You're still not paying attention to what I'm saying, which is that journals are expected to catch the easy errors, not make an absolute determination of truth. I said this in my first comment on this thread.
Re:The blogger who spotted problems in Dana-Farber cancer pa... by ethanbond on Saturday January 27, 2024 @08:31AM (#39155385)
Remarkable you can have this opinion about what should happen and then one comment later run into the exact reason that this is a terrible idea.

Failing to find an error doesn’t mean the paper is true. Correct! That’s why passing peer review isn’t and cannot be a badge of truthfulness.