News for hackers. Stuff that matters.
Re:Why isn't preprint review being adopted? by matwood on Sunday March 24, 2024 @04:57PM (#39810460)
You actually touch on an interesting point. Is peer review still necessary? When there is limited journal space, sure. But now that we have effectively unlimited space, let it be reviewed when people go to cite it. We've already seen a lot of peer reviewed papers be retracted later, so should we just accept the reality?
Re:Why isn't preprint review being adopted? by gwerbret on Sunday March 24, 2024 @05:21PM (#39810625)
Peer review is the backbone of journals, and (some) journals serve an absolutely critical function in academia: they increase the signal:noise ratio. Most of what is published is noise; without the curation provided by (certain) journals, anything of significance is very likely to be drowned out.

As a casual example in the biomedical sciences, the Journal of Biological Chemistry has an output of ~30,000 pages per year, most of which is 'noise'. That's just ONE journal. The journal Cell, on the other hand, has an order of magnitude less, most of which is 'signal'.

EDIT: This is not to say the peer review approach doesn't need work, and lots of it. The whole current approach to research needs an overhaul. I'm just saying it's a bit hasty to throw the baby out with the bath water.

Re:Why isn't preprint review being adopted? by bumby on Sunday March 24, 2024 @07:39PM (#39811530)
Necessary to share ideas? No.

Necessary to share important ideas? Maybe.

One of the aspects of journals that I don't see talked about much here is the curation of articles. Saying (good) journals aren't necessary is like saying journalists aren't necessary to get your news in the internet era. Not all information is necessarily good, and we often rely on systems/people to help winnow the amount we have to sift through. This is important in a society where the sheer amount of information developed far outpaces what we can consume in several lifetimes.

Re:Why isn't preprint review being adopted? by llm_trw on Sunday March 24, 2024 @08:01PM (#39811660)
It's more like saying that maybe we don't need the NYT, the people who lied about Iraq: https://www.theguardian.com/world/2004/may/27/media.iraq, to get good quality journalism when the internet exists.

Journals are in the same boat. Any good they have done is dwarfed by the few 'honest mistakes', like Alzheimer's cabal: https://www.statnews.com/2019/06/25/alzheimers-cabal-thwarte..., which have done more to stifle science than anyone since Stalin picking which biological theories are socialist enough to be true.

Re:Why isn't preprint review being adopted? by bumby on Sunday March 24, 2024 @08:13PM (#39811727)
I think you're conflating the notion of perfect curation. I don't think anyone is claiming that journals or journalists are infallible. They certainly make m, although I don't think I agree that the good is "dwarfed" by them, except when used as a direct attempt to undermine credibility by adversaries.

Put differently, do you think bad ideas spread more easily with the internet? The current research seems to think so, and if you agree, I don't see how that mitigates the spread of bad information. If anything, it exacerbates it.

Re:Why isn't preprint review being adopted? by llm_trw on Sunday March 24, 2024 @08:59PM (#39811965)
I think that every bad idea spread by the internet dwarfs the harms done by ideas spread by government. Every issue with the middle east over the last 20 years is the direct result of a crusade launched on a lie enthusiastically repeated by the same people claiming the moral high ground today.
Re:Why isn't preprint review being adopted? by adastra22 on Sunday March 24, 2024 @05:24PM (#39810653)
No, it is not necessary. Just as it was not necessary for the first 300 years of the Royal society’s existence. “Peer review” used to be just a stand in phrase for the marketplace of ideas—seeing how your peers evaluate your work in a public process of getting published and sending letters to the editor. Only in living memory has it been warped into this pre-publication gatekeeping process.
Re:Why isn't preprint review being adopted? by freedomben on Sunday March 24, 2024 @05:23PM (#39810645)
Peer review is an important signal to potential citers. If everyone has to fully read and understand every paper in order to (responsibly) cite it, there's gonna be a lot less research. Given the exposure of how much bad research there is, maybe there does need to be a slowing and focus on quality, but I think we still need peer review, although it definitely need to be reformed somehow because it's clearly broken.

We need to take a serious look at the incentive structure in academia because it's not guaranteeing the scientific results that we expected it to. I don't think we should just abandon the system though.

Re:Why isn't preprint review being adopted? by llm_trw on Sunday March 24, 2024 @08:02PM (#39811668)
Peer review can be very simply replaced by skimming the abstract. If the abstract doesn't make sense the paper doesn't make sense either.

All the disciplines which use arxiv as their main journal are doing well enough without peer review.

Re:Why isn't preprint review being adopted? by dazed_confused on Sunday March 24, 2024 @05:16PM (#39810592)
Yes. The crap people publish even with peer-reviews is bad enough. A few subfields in CS have bitten a couple times by relying on non peer reviewed ecosystems.