Several independent audits (not counting Pangram's own research, of course) all finding extremely low or zero false positive rates and pretty good recall:
https://epoch.ai/data-insights/ai-detectors-false-negatives
https://arxiv.org/html/2608.11256
https://link.springer.com/article/10.1007/s40979-026-00226-w
Also, the fact that when a public bounty was placed on a known-human text of >1000 words that Pangram falsely flags as having a >75% AI content, the only example anyone found was this one: https://twitter.com/Nition/status/2122500484007395664
> Pangram is just routinely spammed in every other HN thread, but as far as I can tell, it's not much more reliable than rand()%2
As far as you can tell... based on what?
> It doesn't and cannot work in general case. It's trivial to make LLM write in any style[...]
If you asked me two years ago whether AI detection was possible, I'd have answered no on these grounds. It turns out, however, that RLHF as it's currently done causes LLMs to collapse to very specific styles, and labs are either unwilling or unable to figure out a way to fix it, which is why both humans and models are in fact able to detect AI-written text. Base models don't have this style collapse and indeed aren't detected by Pangram, but very few people are willing to use base models. Similarly, finetuning a model to write in a specific style would probably suffice to evade detection, but very few people are willing to put in this much effort. The result is that it's hard to be sure that any given text isn't AI-written, but most AI-written text is extremely blatant.
Pangram is just routinely spammed in every other HN thread, but as far as I can tell, it's not much more reliable than rand()%2, and that's only for cases where the "author" doesn't even care to hide the text was generated.
It doesn't and cannot work in general case. It's trivial to make LLM write in any style, or have it prorduce arbitrary number of typos and grammar mistakes. Most people just don't bother to add an extra line to the prompt.