News for hackers. Stuff that matters.
Re:Can Peer Review Survive AI-Generated Text? [pdf] by bennettnate5 on Wednesday November 18, 2026 @09:45AM (#50546942)
The computer security research community is strongly against using AI detectors, in part because the evidence has come against their being at all useful at scale [1]. Far too many false positives and negatives that the whole thing effectively becomes a witch-hunt test.

[1] https://ieeexplore.ieee.org/stampPDF/getPDF.jsp?tp=&arnumber...

Re:Can Peer Review Survive AI-Generated Text? [pdf] by azan_ on Wednesday November 18, 2026 @09:55AM (#50547036)
> Our evaluation not only finds that performance varies wildly across AIGT detectors (e.g., FPRs between 0.05 % and 68.6 % and FNRs ranging between 0.3 % and 99.6 %).

So it appears that there are good detectors that can be used and bad ones that shouldn’t.

Re:Can Peer Review Survive AI-Generated Text? [pdf] by Borealid on Wednesday November 18, 2026 @10:02AM (#50547129)
That set of statistics does not tell you whether there are good detectors or not.

If the low FPR and the low FNR are the same detector, that's good. But it's more likely one detector has a high FNR and a different detector a high FPR, and the texts causing errors may be correlated across detectors making a "best of both worlds" multipass approach difficult or impossible.

Re:Can Peer Review Survive AI-Generated Text? [pdf] by azan_ on Wednesday November 18, 2026 @11:05AM (#50547996)
Low fpr with avg fnr is still very useful.
Re:Can Peer Review Survive AI-Generated Text? [pdf] by bheadmaster on Wednesday November 18, 2026 @10:09AM (#50547240)
South Park perfectly depicted the nature AI detectors:

https://youtu.be/kVzUGRaWVT4

Re:Can Peer Review Survive AI-Generated Text? [pdf] by tmpz22 on Wednesday November 18, 2026 @10:28AM (#50547489)
Why do we invest 100s of Billions in the belief AGI is possible, yet an AI detector which protects the integretity of critical research is beyond our grasp?

Quitter.

Re:Can Peer Review Survive AI-Generated Text? [pdf] by dale_glass on Wednesday November 18, 2026 @10:33AM (#50547541)
Because it's an impossible problem. AI generates just words, and both AI and people use the same words. In fact both learn from each other. AI learns from people and eventually people will pick up patterns from AI.

Also, in many cases, a false positive hurts the innocent more than the cheaters. Many cheaters are already taking failure into account, while those innocent do not.

Re:Can Peer Review Survive AI-Generated Text? [pdf] by OneManyNone on Wednesday November 18, 2026 @10:39AM (#50547631)
I cannot read the entirety of this article, but nowhere in the abstract does it mention Pangram, which has repeatedly been shown in many independent studies to have a false positive rate of less than 1 in 10,000; by some accounts it is now down to 1 in 20,000.

Obviously there are many bad detectors that shouldn't be used, but it's so bizarre when studies like this leave out extremely high-quality detectors. For some reason there's just a segment of people who are dead-set on letting the world get polluted with bad writing.

Re:Can Peer Review Survive AI-Generated Text? [pdf] by MakisH on Thursday November 19, 2026 @03:33AM (#50557497)
The study in this paper does use Pangram, and they state that the version they use has much lower false positive rate than earlier versions. They also state that this is only meant as a signal that the authors will need to provide evidence against.