cor_NEEL_ius

The validation point is real. We tested this with AI presentation tools specifically - gave 6 of them the same prompt and fact-checked every claim against primary sources. Best accuracy was 44%. Most were under 20%.

The pattern was consistent: the tools produce confident, well-formatted output that looks thoroughly researched. But more than half the statistics were either distorted or completely fabricated. The worst part was finding the same fake stats appearing across multiple tools - not because they independently verified anything, but because they all absorbed the same bad data from training.

The productivity gains from AI are real, but so is the validation cost. People just aren't accounting for it.

The scarier version of this problem is what I've been calling "zombie stats" - numbers that get cited across dozens of sources but have no traceable primary origin.

We recently tested 6 AI presentation tools with the same prompt and fact-checked every claim. Multiple tools independently produced the stat "54% higher test scores" when discussing AI in education. Sounds legit. Widely cited online. But when you try to trace it back to an actual study - there's nothing. No paper, no researcher, no methodology.

The convergence actually makes it worse. If three independent tools all say the same number, your instinct is "must be real." But it just means they all trained on the same bad data.

To your question about claim-level verification: the closest I've seen is attaching source URLs to each claim at generation time, so the human can click through and check. Not automated verification, but at least it makes the verification possible rather than requiring you to Google every stat yourself. The gap between "here's a confident number" and "here's a confident number, and here's where it came from" is enormous in practice.