aionwikipedia

the extent to which grokipedia is a disaster is, in a way, kind of fascinating:

- system prompts leaked all over the place ("Never speculate or make up information. Do not claim anything that is not verifiably true" etc) as well as text like "To complete the task, I will write the content using the ground truth provided, and use plausible citations based on common sources."

- function calls failing in plaintext resulting into article text trailing off into "I am a disgrace"-style infinite loops

- lax quality filter on article requests, causing articles to proliferate on random arxiv identifiers like "210608603" (which it attempts to write about as numbers, flailingly), SEO keywords, and people's custom AI erotica prompts

most papers like this are flagged by AI detectors because, by your own admission, AI was used to generate the final output. what exactly did you expect here?

even if your writing is influenced by "the AI style," it would likely only show up in patches, a word or rhetorical flourish maybe, but AI-generated text doesn't "sound like AI" in patches, it does so consistently across the entirety of the output. it's very hard for a person to maintain the syntactical patterns of AI writing over an extended period of time unless they know exactly what those are -- probably more knowledge than any person currently has, really -- and are applying a level of detail to each clause comparable to forging a painting.

there's also the fact that "the AI style" has changed over time. for instance, the word "delve" is notorious as an "AI sign," which it was up until mid-2024, at which point it dropped off sharply and has now basically disappeared from LLM output. so if someone happened to pick that up due to reading it everywhere, their writing is now less characteristic of AI, not more.

Yeah the "more than a paragraph or two" is key here. Indicators of AI writing work both ways; the more text you write, the more likely you are to "slip" and use some phrasing or syntactical constructions uncommon in LLM output. (This is why AI detectors perform worse on shorter excerpts.)

I posted this elsewhere, but convincingly, consistently "writing like AI" and never slipping once takes an amount of knowledge and skill analogous to art forgery. Except that with art forgery you can at least make millions of dollars off it.

This is a myth. At least one study (Juzek/Ward) has shown that these stylistic patterns do not appear nearly as often either in well-known datasets of English language -- including datasets restricted to specific dialects of English. They don't even appear as often in text generated by raw language models. When they start showing up is after the model has undergone RLHF. Think of the Fermi paradox: if there are all these people who write like AI, then where are they?

AI writing also tends to show these indicators over and over, consistently, over passages of text. It is very hard for humans, even if they are really familiar with AI writing, to be that consistent, and almost impossible for them to be that consistent for more than a sentence or two. Writing a long blog post by hand that is believably "AI-written" takes the amount of purposeful skill you'd need to forge an entire painting or ancient document.

The problem is that people either look for the wrong things, look for obsolete things ("delve" is dead and modern LLMs have killed it), or extrapolate things from indicators that are extremely narrow and specific.

If you know what you're looking for -- and no, that does not mean em-dashes, they're a low-signal indicator and people get precious about them -- your false positive rate can get pretty low. See Russell et al, "People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text."

Anecdotally speaking, I'm heavily involved in AI cleanup work on Wikipedia. At this point, I can read article text typical of AI, guess within 6-12 months when that edit was made based on differences in LLM output over the years, and usually be right. (e.g., "delve" was common through early 2024 until GPT-4o killed it; I've seen theories that it's not a GPT-4o thing but rather the result of people avoiding the word when it became a meme, but its frequency dropped off hard even in edits that clearly were not reviewed or edited at all.) Given that Wikipedia has accumulated billions of edits over 25+ years, that says a lot.

"Usually revisions are approved manually by real people."

almost never the case besides a select few articles that get heavily vandalized and require all edits to be approved. otherwise, anyone can edit Wikipedia at any time, which famously is the point of Wikipedia