How we measured AI writing across arXiv, and where the measurement breaks

Re:How we measured AI writing across arXiv, and where the me... by economistbob on Monday July 20, 2026 @01:06PM (#48981592)

The problem is that humans start with credulity. AI hallucinates and makes up stuff some percentage of the time. Humans are not "default deny" when given information. Unleashing that was a disservice to mankind and has created a future filled with lies and people who are confident in them.

[ Parent | on HN ]

Re:How we measured AI writing across arXiv, and where the me... by RodgerTheGreat on Monday July 20, 2026 @01:39PM (#48982100)

LLMs are in essence an attack on the concept of written language, harvesting and dissolving the social contract that underpins it; it is no longer safe to assume that text has intent, let alone content, simply because it is grammatically well-formed.

Perhaps the most darkly amusing consequence of this particular mania is that by poisoning the majority of our information environment with hallucinated slop, we have likely crippled the next several generations of machine-learning techniques before they're even invented! Small, locally-hostable LLMs will rattle along spewing spam long after the broader "genai bubble" pops, and building clean training datasets will permanently be more difficult and expensive.

[ Parent | on HN ]

Re:How we measured AI writing across arXiv, and where the me... by jrm4 on Monday July 20, 2026 @01:13PM (#48981713)

Oh, to be technically correct:

AI hallucinates and makes up stuff 100% percent of the time. Never been a fan of that word for this.

Again, I fail to see the problem here that isn't solved by careful reading WHICH IS WHAT PEOPLE SHOULD BE DOING ANYWAY. I would like to see room for AI disclosure, maybe a statement of "this is how much AI I used."

But this blanket X% of this looks like AI? Again, so what?

[ Parent | on HN ]

Re:How we measured AI writing across arXiv, and where the me... by jsrozner on Monday July 20, 2026 @01:31PM (#48982012)

If as a primary school student you had needed to reverify every fact /experiment that was presented in your textbooks, you would not have gotten very far.

Trust is very important to human progress.

[ Parent | on HN ]

Re:How we measured AI writing across arXiv, and where the me... by jrm4 on Tuesday July 21, 2026 @11:38AM (#48993758)

Absolutely. Which is why what I think I'm saying is that -- it's not ncessarily important who or what WROTE the paper, it's simply important that human-in-the-loop is strong.

[ Parent | on HN ]

Re:How we measured AI writing across arXiv, and where the me... by jsrozner on Wednesday July 22, 2026 @08:03PM (#49015160)

Right, but even careful reading cannot validate the experimental setup.

[ Parent | on HN ]

Re:How we measured AI writing across arXiv, and where the me... by JSR_FDED on Monday July 20, 2026 @01:29PM (#48981970)

I find the fact that 39% of papers use a style that signals lack of effort somewhat worrisome. Even if that’s 100% wrong, and they’re all high effort papers, the fact that they give off the same aura as low effort papers is a problem for the authors.

[ Parent | on HN ]