News for hackers. Stuff that matters.
Re:Beware the Man of Many Studies by empiko on Wednesday June 07, 2023 @10:48AM (#36227469)
Peer review is not a certificate of truthfulness, peer review just says that a grad student reading the paper did not found anything too suspicious there and it looks like a typical paper in that field.
Re:Beware the Man of Many Studies by svnt on Wednesday June 07, 2023 @11:22AM (#36227987)
Peer review, at its core, is a social consensus process. It can work, but it is structurally inclined to propagate agreement.

I wonder if it serves to reinforce this problem, as those more motivated to reinforce certain literatures or perspectives will surface those repeatedly in reviews.

Re:Beware the Man of Many Studies by all2 on Wednesday June 07, 2023 @12:51PM (#36229512)
There's a popular work that comes to this conclusion in its beginning chapters [0]. The basic premise is that the act of science and knowledge acquisition cyclically devolves into ideology and is then disrupted. Typically those who initially disrupt a scientific dogma are not treated well. Eventually the old guard literally dies off and the new ideas can begin to take hold.

[0] The Structure of Scientific Revolutions, T. Kuhn

Re:Beware the Man of Many Studies by consilient on Wednesday June 07, 2023 @04:46PM (#36233306)
> The basic premise is that the act of science and knowledge acquisition cyclically devolves into ideology and is then disrupted.

This can happen of course, but it's not really what Kuhn is talking about. Kuhnian paradigm shifts are to science what massive refactorings are to a codebase: the problem isn't that the old code was wrong (though it might also be wrong), it's that it can't be extended to meet new requirements. And while some of us are better at writing extensible code than others, no one gets it right every single time.

Re:Beware the Man of Many Studies by some_random on Wednesday June 07, 2023 @11:02AM (#36227684)
A huge part of the problem is that the layperson is taught otherwise.
Re:Beware the Man of Many Studies by hospitalJail on Wednesday June 07, 2023 @11:10AM (#36227815)
2nd grade we learned the scientific method. I don't remember peer reviewed being a thing.

Replication was critical.

Also, I've seen peer reviewed papers with atrocious stuff in them, unless I know the peer, it doesn't mean much to me. Give me 100 independent groups coming to the same conclusions. That has far more weight than a few of your friends/professors reading your paper.

Re:Beware the Man of Many Studies by biomcgary on Wednesday June 07, 2023 @12:45PM (#36229449)
Not even that. I was a grad student that reviewed a paper submitted to a well-known journal from a heavyweight in my field. The paper's own data showed that the authors were reporting 95% noise. My advisor rejected the paper only to see it published a few months later (in another famous journal) after the authors removed the data that allowed us to detect the noise throughout the data.