Re:A Failure of Academic Quality Control [pdf]
by mkl on Monday September 03, 2018 @03:24AM
(#17901024)
The title is literally "A Failure ...". One single failure, and that's what it describes. There are only a handful of paragraphs about generalising, and these are knowingly subjective; two of them start with "We believe".
Typically at conferences you have a area chair who is selected due to some connection with organizer and then this area chair choses other people is his/her network to be reviewers for submissions they receive. So from top to bottom, the process is driven by social connections and very often used as paying back past favors or create a future favor. I have coined even term for this: favor economy.
One way to fix this would be to create a metric for reviewer like r-index in similar line to h-index which rewards reviewer for choosing high impact papers and somehow penalized them to pass on good papers. On a simplest level, r-index could simply be same as h-index for papers reviewed instead of authored. However I think there ought to be better measure.