Re:AI intensifies fight against ‘paper mills’ that churn out...
by JoshuaJB on Wednesday May 31, 2023 @06:26PM
(#36145234)
Although there is a type of peer review that includes redoing experiments and analysis: artifact evaluation. All of the top conferences in my field (real-time embedded systems) include this as an opt-in option, and papers get a special badge if they also pass artifact evaluation. I strongly believe that other fields in computer science would benefit by including and normalizing this process.
Besides the reproducibility benefits, artifact evaluation forces documentation of the experiments and process; I've found this enormously useful when on-boarding new students to an existing project.
Re:AI intensifies fight against ‘paper mills’ that churn out...
by Balgair on Thursday June 01, 2023 @10:19AM
(#36151366)
I'm nearly certain that your's is the only field that does anything like re-experimentation then. I'm in biotechy fields and it's a totally different beast out here man.
Besides the reproducibility benefits, artifact evaluation forces documentation of the experiments and process; I've found this enormously useful when on-boarding new students to an existing project.