News for hackers. Stuff that matters.
Re:Can Peer Review Survive AI-Generated Text? [pdf] by patrickmay on Wednesday November 18, 2026 @09:06AM (#50546486)
Abstract, for those not interested in reading TFA:

"Large language models now write papers and referee reports in economics. The field must develop best practices for handling AI-generated text in the publication process. Otherwise, peer review is in serious danger of collapsing. We propose that AI detectors should become a standard tool of the profession. Our experiment shows that they flag referee reports written by AI, not reports that AI merely proofread or helped edit.

Aided by a simple model, we argue that a paper flagged as AI-generated should trigger a closer reading and can be rejected at the first mistake, and that an AI-generated referee report adds nothing that editors could not get by running the prompt themselves. Such reports are unacceptable: they damage authors and journals and should be treated like plagiarism. Individual restraint has already shown its limits, and the profession needs to develop norms to solve this collective action problem. We close with a practical guide for authors, referees, and editors."

Re:Can Peer Review Survive AI-Generated Text? [pdf] by p-e-w on Wednesday November 18, 2026 @09:12AM (#50546549)
> Aided by a simple model, we argue that a paper flagged as AI-generated should trigger a closer reading and can be rejected at the first mistake

“We propose a witch hunt fueled by unreliable, unaccountable, proprietary detection technology that’s roughly equivalent to tea leaf reading and can be fooled by trivial rewrite templates that anyone can find with a 5-second web search.”

Re:Can Peer Review Survive AI-Generated Text? [pdf] by pwndByDeath on Wednesday November 18, 2026 @09:18AM (#50546623)
I found it more ironic that in order to save a burdened system we'll add a step that demands a heavier burden to the system.
Re:Can Peer Review Survive AI-Generated Text? [pdf] by javier123454321 on Wednesday November 18, 2026 @09:41AM (#50546889)
Just throw more compute at it is the answer for everything for the AI pilled.
Re:Can Peer Review Survive AI-Generated Text? [pdf] by itomato on Wednesday November 18, 2026 @09:26AM (#50546715)
The more I get serious about this, the more I want to see Machines produce Machine-native output. I don't want crossover or doubt. I want high confidence that machine output:

- is optimized for machine input

- is convertible to human-native symbols, context

- includes metadata and proof-of-custody/origin

- leans into its role as an evolving language

- is necessarily terse and token-efficient

Re:Can Peer Review Survive AI-Generated Text? [pdf] by ramses0 on Wednesday November 18, 2026 @09:45AM (#50546937)
Basically: `unzaip output.zaip.gpg.sig --lang=EN-us --include-prompt`

`zip` => `zAIp`

It gets around some of my general complaints about "checking generated artifacts into version control", really interesting idea!

Re:Can Peer Review Survive AI-Generated Text? [pdf] by itomato on Thursday November 19, 2026 @06:48AM (#50558920)
More like IBM or Dec 'SQUOZE'
Re:Can Peer Review Survive AI-Generated Text? [pdf] by bennettnate5 on Wednesday November 18, 2026 @09:45AM (#50546942)
The computer security research community is strongly against using AI detectors, in part because the evidence has come against their being at all useful at scale [1]. Far too many false positives and negatives that the whole thing effectively becomes a witch-hunt test.

[1] https://ieeexplore.ieee.org/stampPDF/getPDF.jsp?tp=&arnumber...

Re:Can Peer Review Survive AI-Generated Text? [pdf] by azan_ on Wednesday November 18, 2026 @09:55AM (#50547036)
> Our evaluation not only finds that performance varies wildly across AIGT detectors (e.g., FPRs between 0.05 % and 68.6 % and FNRs ranging between 0.3 % and 99.6 %).

So it appears that there are good detectors that can be used and bad ones that shouldn’t.

Re:Can Peer Review Survive AI-Generated Text? [pdf] by Borealid on Wednesday November 18, 2026 @10:02AM (#50547129)
That set of statistics does not tell you whether there are good detectors or not.

If the low FPR and the low FNR are the same detector, that's good. But it's more likely one detector has a high FNR and a different detector a high FPR, and the texts causing errors may be correlated across detectors making a "best of both worlds" multipass approach difficult or impossible.

Re:Can Peer Review Survive AI-Generated Text? [pdf] by azan_ on Wednesday November 18, 2026 @11:05AM (#50547996)
Low fpr with avg fnr is still very useful.
Re:Can Peer Review Survive AI-Generated Text? [pdf] by bheadmaster on Wednesday November 18, 2026 @10:09AM (#50547240)
South Park perfectly depicted the nature AI detectors:

https://youtu.be/kVzUGRaWVT4

Re:Can Peer Review Survive AI-Generated Text? [pdf] by tmpz22 on Wednesday November 18, 2026 @10:28AM (#50547489)
Why do we invest 100s of Billions in the belief AGI is possible, yet an AI detector which protects the integretity of critical research is beyond our grasp?

Quitter.

Re:Can Peer Review Survive AI-Generated Text? [pdf] by dale_glass on Wednesday November 18, 2026 @10:33AM (#50547541)
Because it's an impossible problem. AI generates just words, and both AI and people use the same words. In fact both learn from each other. AI learns from people and eventually people will pick up patterns from AI.

Also, in many cases, a false positive hurts the innocent more than the cheaters. Many cheaters are already taking failure into account, while those innocent do not.

Re:Can Peer Review Survive AI-Generated Text? [pdf] by OneManyNone on Wednesday November 18, 2026 @10:39AM (#50547631)
I cannot read the entirety of this article, but nowhere in the abstract does it mention Pangram, which has repeatedly been shown in many independent studies to have a false positive rate of less than 1 in 10,000; by some accounts it is now down to 1 in 20,000.

Obviously there are many bad detectors that shouldn't be used, but it's so bizarre when studies like this leave out extremely high-quality detectors. For some reason there's just a segment of people who are dead-set on letting the world get polluted with bad writing.

Re:Can Peer Review Survive AI-Generated Text? [pdf] by MakisH on Thursday November 19, 2026 @03:33AM (#50557497)
The study in this paper does use Pangram, and they state that the version they use has much lower false positive rate than earlier versions. They also state that this is only meant as a signal that the authors will need to provide evidence against.