Scientists aim to pull peer review out of the 17th century

Re:Scientists aim to pull peer review out of the 17th century by allthenews on Monday February 26, 2018 @12:11PM (#16466892)

Sounds like a movement toward crowdsourcing, long overdue for our increasingly broken scientific process.

If the HN experiment has worked for so long, surely, with proper controls on participation and interaction, there's no need to worry about any kind of "Wild Wild West," as the author put it.

What I wonder is whether anonymity would be better or worse for some kind of semi-public peer review forum.

[ Parent | on HN ]

Re:Scientists aim to pull peer review out of the 17th century by mlspector on Monday February 26, 2018 @02:01PM (#16467954)

I'm one of the main developers of OpenReview.net, and we're trying to do something similar to APPRAISE as described in the OP. Right now we mainly focus on the machine learning research community, and one conference in particular - ICLR - has been experimenting with varying degrees of anonymity in their peer review process.

We've had some preliminary discussions about evaluating the impact of anonymity on peer review "quality", but so far the proper way to analyze the data isn't quite clear to us.

We do have an open API that allows the general public to access the data, but I'm embarrassed to say that our documentation is very sparse right now. If anyone reading this is interested in exploring the data, shoot us an email at [email protected] and we can help you get started.

https://openreview.net/about

[ Parent | on HN ]

Re:Scientists aim to pull peer review out of the 17th century by forapurpose on Monday February 26, 2018 @10:50PM (#16470948)

> If the HN experiment has worked for so long ...

HN is relatively good for an Internet forum, not for scientific literature. It's not even good professional IT work product - that's not an insult; it's not what the forum is intended to be. I'd be very concerned if the quality of scientific research and review dropped to this level.

[ Parent | on HN ]

Re:Scientists aim to pull peer review out of the 17th century by kickout on Monday February 26, 2018 @12:15PM (#16466935)

Anonymity would be better in my opinion (to start off with). Way too much name recognition, and "good old boys club" dynamics happening in way too many areas of science i follow (genetics, agriculture, prediction)

[ Parent | on HN ]

Re:Scientists aim to pull peer review out of the 17th century by neuromantik8086 on Monday February 26, 2018 @01:14PM (#16467509)

Pseudonymity might also be a good option (although it also has its pitfalls):

https://twitter.com/jaipelai/status/900513036552859648

https://news.ycombinator.com/item?id=15918851

[ Parent | on HN ]