News for hackers. Stuff that matters.
Re:Collusion rings threaten the integrity of computer scienc... by geraltofrivia on Wednesday May 26, 2021 @06:18PM (#27296798)
The playing field where one can share research findings is far more uneven, more tilted in favor of the prestigious labs and individuals more than the current field of conferences and the blind review process.

Findings and papers from well known individuals (read: twitter accounts) do get far more attention , more citations. Of course, one can argue that, broadly, well known labs and individuals are wel known because of their tendency to do great work, write better papers. And that’s true. However, the above still holds, in my experience as PhD student in ML. Anecdotally, I have seen instances where a less interesting paper from a renowned lab got more attention and eventually more citations than a better paper accepted at the same venue by a less renowned lab on the same topic.

Re:Collusion rings threaten the integrity of computer scienc... by cycomanic on Thursday May 27, 2021 @02:49AM (#27300155)
I completely agree with you and this is a significant issue, because it makes it hard for someone not from one of the established places to get research attention.

I would also argue that with the increased importance of the (mostly) commercial "high-impact" journals this has become worse. I know that some of the professional (non-expert) editors of these journals specifically look at the citation counts of authors before accepting to send them out to review, because their main aim is to get people reading the articles, not necessarily good science.

Re:Collusion rings threaten the integrity of computer scienc... by geraltofrivia on Thursday May 27, 2021 @03:52AM (#27300448)
This is objectively worse. Commercial journals need to be bygone. I feel that the publishing venues in the field of machine learning are a great example of non commercial, (relatively) transparent, open access publications. Look at ICLR, ICML, ACL, NeurIPS, for instance. All of their proceedings are open access. Most papers their are available on arXiv as well (which has its pros and cons both that I'd rather not get into here). Many of them also have an open review process wherein the reviewers post their reviews for all to see in a forum like interface, allowing frequent back and forth with the authors. In the best case, its more akin to assist the authors refine their potentially sound findings, a dynamic process where the paper may get revised a couple times before acceptance.

I believe journals need to adapt the openreview process. There are obvious parallels in the lifecycle of a journal submission (from submission to acceptance) and the openreview process, except that the latter is accelerated (for better or for worse).

Re:Collusion rings threaten the integrity of computer scienc... by cycomanic on Thursday May 27, 2021 @06:44AM (#27301330)
Yes, the cloud that the large commercial journals have over the academic process is really worrying. A publication in Nature can mean the difference between tenure and no tenure or getting a grant or no grant.

I have to say that I find the review process of the copernicus journals very interesting. You can see a description here: [1]. Unfortunately I don't work in a related field otherwise I would have published there already.

[1] https://www.atmospheric-chemistry-and-physics.net/peer_revie...

Re:Collusion rings threaten the integrity of computer scienc... by BlueTemplar on Thursday May 27, 2021 @08:58AM (#27302513)
Yeah, I was wondering : has someone tried to compare citations of papers by researchers that do not tweet / blog to those who do ?
Re:Collusion rings threaten the integrity of computer scienc... by geraltofrivia on Thursday May 27, 2021 @09:34AM (#27302888)
Yep.

> We find considerable evidence that, overall, article citations are positively correlated with tweets about the article, and we find little evidence to suggest that author gender affects the transmission of research in this new media

https://journals.plos.org/plosone/article?id=10.1371/journal...

(I've only skimmed the paper, a few months ago).