Re:GIMP now has UI tests
by stratos123 on Friday November 20, 2026 @04:17AM
(#50572163)
> What record? Where is it? What is it based on?
Several independent audits (not counting Pangram's own research, of course) all finding extremely low or zero false positive rates and pretty good recall:
Also, the fact that when a public bounty was placed on a known-human text of >1000 words that Pangram falsely flags as having a >75% AI content, the only example anyone found was this one:
https://twitter.com/Nition/status/2122500484007395664
> Pangram is just routinely spammed in every other HN thread, but as far as I can tell, it's not much more reliable than rand()%2
As far as you can tell... based on what?
> It doesn't and cannot work in general case. It's trivial to make LLM write in any style[...]
If you asked me two years ago whether AI detection was possible, I'd have answered no on these grounds. It turns out, however, that RLHF as it's currently done causes LLMs to collapse to very specific styles, and labs are either unwilling or unable to figure out a way to fix it, which is why both humans and models are in fact able to detect AI-written text. Base models don't have this style collapse and indeed aren't detected by Pangram, but very few people are willing to use base models. Similarly, finetuning a model to write in a specific style would probably suffice to evade detection, but very few people are willing to put in this much effort. The result is that it's hard to be sure that any given text isn't AI-written, but most AI-written text is extremely blatant.
Several independent audits (not counting Pangram's own research, of course) all finding extremely low or zero false positive rates and pretty good recall:
https://epoch.ai/data-insights/ai-detectors-false-negatives
https://arxiv.org/html/2608.11256
https://link.springer.com/article/10.1007/s40979-026-00226-w
Also, the fact that when a public bounty was placed on a known-human text of >1000 words that Pangram falsely flags as having a >75% AI content, the only example anyone found was this one: https://twitter.com/Nition/status/2122500484007395664
> Pangram is just routinely spammed in every other HN thread, but as far as I can tell, it's not much more reliable than rand()%2
As far as you can tell... based on what?
> It doesn't and cannot work in general case. It's trivial to make LLM write in any style[...]
If you asked me two years ago whether AI detection was possible, I'd have answered no on these grounds. It turns out, however, that RLHF as it's currently done causes LLMs to collapse to very specific styles, and labs are either unwilling or unable to figure out a way to fix it, which is why both humans and models are in fact able to detect AI-written text. Base models don't have this style collapse and indeed aren't detected by Pangram, but very few people are willing to use base models. Similarly, finetuning a model to write in a specific style would probably suffice to evade detection, but very few people are willing to put in this much effort. The result is that it's hard to be sure that any given text isn't AI-written, but most AI-written text is extremely blatant.