As for the reason why it happens in conferences, I think it may actually be a different set of reviewers. While journal reviewers are going to be conference reviewers, I don't think the other way around is true. I think conferences tend to just have a larger number of shitty reviewers (as well as more shitty submissions). And as you note, it is quite easy to misunderstand a work, doubly so when you're reading under a time constraint. It just makes for a noisy process, especially when reviewers view their job as to reject (not improve). I just think it is a bad system with a bad premise that can't really be fixed. For conference reviewing, I always try to write what would change my mind and if I think the authors should resubmit to another venue. But even reviewing I don't feel authors get a fair shot at responding. They can't address all my comments while addressing others in a single page.
Re:Bad numbers in the “gzip beats BERT” paper?
by godelski on Monday July 17, 2023 @05:38PM
(#36764450)
Yeah, honestly the only reason I submit to conferences now is because my advisor asks me to. If it was up to me I would submit exclusively to journals or just to arxiv/open review directly. I think I'll do this when I graduate (soon).
My second journal paper got rejected with encouragement to resubmit. Half the reason for that was because the reviewer had, I think, genuinely misunderstood the description of an experiment, so I re-wrote it in painstaking detail. I had a long section where I hammered out a proof of complexity spanning three pages, with four lemmas and a theorem, and the reviewer waded through all that like a hero, and caught errors and made recommendations for improvement. They made a new round of recommendations when I resubmitted. That paper took three rounds of revisions to publish (reject, resubmit, accept with minor revisions) but it got 80% better every time I had to revise it. I wish there was another couple of rounds! It was exhausting, and I bet much more so to the reviewer, but it was 100% worth it.
And yeah, I absolutely do my best to review like that myself. Even in conferences, which probably seems really weird to authors. But, hey, be the change you want to see.