“We propose a witch hunt fueled by unreliable, unaccountable, proprietary detection technology that’s roughly equivalent to tea leaf reading and can be fooled by trivial rewrite templates that anyone can find with a 5-second web search.”
- is optimized for machine input
- is convertible to human-native symbols, context
- includes metadata and proof-of-custody/origin
- leans into its role as an evolving language
- is necessarily terse and token-efficient
`zip` => `zAIp`
It gets around some of my general complaints about "checking generated artifacts into version control", really interesting idea!
[1] https://ieeexplore.ieee.org/stampPDF/getPDF.jsp?tp=&arnumber...
So it appears that there are good detectors that can be used and bad ones that shouldn’t.
If the low FPR and the low FNR are the same detector, that's good. But it's more likely one detector has a high FNR and a different detector a high FPR, and the texts causing errors may be correlated across detectors making a "best of both worlds" multipass approach difficult or impossible.
Quitter.
Also, in many cases, a false positive hurts the innocent more than the cheaters. Many cheaters are already taking failure into account, while those innocent do not.
Obviously there are many bad detectors that shouldn't be used, but it's so bizarre when studies like this leave out extremely high-quality detectors. For some reason there's just a segment of people who are dead-set on letting the world get polluted with bad writing.
"Large language models now write papers and referee reports in economics. The field must develop best practices for handling AI-generated text in the publication process. Otherwise, peer review is in serious danger of collapsing. We propose that AI detectors should become a standard tool of the profession. Our experiment shows that they flag referee reports written by AI, not reports that AI merely proofread or helped edit.
Aided by a simple model, we argue that a paper flagged as AI-generated should trigger a closer reading and can be rejected at the first mistake, and that an AI-generated referee report adds nothing that editors could not get by running the prompt themselves. Such reports are unacceptable: they damage authors and journals and should be treated like plagiarism. Individual restraint has already shown its limits, and the profession needs to develop norms to solve this collective action problem. We close with a practical guide for authors, referees, and editors."