News for hackers. Stuff that matters.

A comment on Arxiv.org reaches a milestone and a reckoning. Parent comment

Re:Arxiv.org reaches a milestone and a reckoning by marginalia_nu on Tuesday January 18, 2022 @10:00AM (#29980129)
> In fact, peer review is often more about novelty and importance rather than rigor. Very well structured research will get rejected if it isn't seen as contributing to the field in a nontrivial way.

This is actually deeply problematic, as a part of what's driving the replication problems is specifically publication bias. If the only thing that gets published are unexpected or noteworthy results, you're selecting for statistical aberrations.

[ Parent | on HN ]
Re:Arxiv.org reaches a milestone and a reckoning by UncleMeat on Tuesday January 18, 2022 @11:28AM (#29981533)
I don't really agree. I think it is valuable to push novel and influential results into visible venues and to reward work of this kind. It is critical to understand how this introduces bias into the process and to consider this when reading a given paper, but I'm not sure that a review system that only considers experimental rigor would be desirable.
[ Parent | on HN ]