See also: the Clarkesworld saga of them being bombarded with mediocre AI-generated short stories. Filtering out bad submissions has always come with the territory, but they're suddenly drowning in them with the advent of LLMs which make it trivial to churn out vaguely story-shaped text on an industrial scale. Human curation can't hope to keep up with an endless barage of AI slop.
If your answer is that Wikipedia and Hacker News still get the recognition and haven't collapsed, then I suggest that there are already examples to learn from that the same idea for Stack Exchange won't work.
I think it's way more likely that poor answers won't mention the usage of LLM's to generate the answer, while good answers aided by LLM's will more often mention it.
Punishing honesty just seems incredibly counterproductive.
Automatic detection is downright dystopian... being censored by an algorithm because it mistook my effort and work for a LLM.
And while I'm not a moderator, as just a user I've flagged over 1,200 answers on Stack Overflow (and several of the smaller communities like Ask Ubuntu) that were subsequently removed. Automatic detection was never the sole criteria that was used to determine if it was AI - It's entirely possible to spot GPT content using multiple methods. I don't publicly talk about most of these, since we do have a group of users (sometimes spammers) who attempt to hide their use and make it more difficult to detect. See some of my additional notes on the topic on https://meta.stackexchange.com/a/389674/902710
Sounds like a job for AI
It's entirely possible to set up the system to prevent it from being flooded by content that moderation can't handle. In fact, StackOverflow has already been largely set up that way, and this will just require just a little more tweaking of the types of existing policies that have already been in place for a long time. People attempting to flood internet forums with low-quality content or outright spam isn't anything new.
However, in practice this sort of approach would likely mean people who don't have anything invested (especially new users) would create multiple accounts to be able to post multiple times.
Rate limiting only works well if there's a stickiness that makes changing accounts more difficult than waiting out the rate limit.
---
While Stack Overflow was set up to handle moderation, the culture evolved to one that disdained any appearance of gate keeping, preservation of any attempt to answer, and that moderation and curation actions on a post were personal attacks on the individual who wrote it.
As tooling was taken away from community moderation and curation it became harder and harder to maintain quality. Additionally, the rule of 90-9-1 (aka The Rule of Participation Inequality - https://www.grazitti.com/blog/the-90-9-1-rule-is-over-its-ti... ) applied to people who are doing moderation and curating means that once it scales above a certain point it becomes impractical if not impossible to curate all of the incoming material.
A little more tweaking may have been possible a decade ago. However, both the culture of people asking questions and the corporate "engagement first" approach have made being a person trying to curate the material fighting against the tide.
There are 3.3k questions that have had a close vote cast that need more people to review them. There have been only 313 reviews today as I write this ( https://stackoverflow.com/review/close/stats ). And that's ignoring the countless thousands of reviews that have timed out.
year close tasks
2016 581,204 https://meta.stackoverflow.com/q/340815
2017 .......
2018 440,336 https://meta.stackoverflow.com/q/378415
2019 318,431 https://meta.stackoverflow.com/q/392550
2020 225,745 https://meta.stackoverflow.com/q/404558
2021 213,104 https://meta.stackoverflow.com/q/415250
2022 96,495 https://meta.stackoverflow.com/q/422885
A trend with community moderation is clearly visible and likely too far to be corrected with tweaking.>This doesn't work at scale... with LLM's all of a sudden there is a huge influx of mediocre content
The GP's answer may not work at scale - however LLM detection doesn't work at all. So the only semi-workable solution is aggressive filtering and banning users who post trash (LLM or not).
Also, there's a need to think about score and trust mechanisms - the same mechanisms which can be used for filtering also provide an incentive for LLM use, is there a way to avoid that?
>When someone posts a answer I assume they actually ran the code, and can verify the result
I wish we lived in a world where this assumption wasn't naive.
For example here on HN we have a rule if you see bot content you don't mention it in the thread. You report it and let the admins decide. Anything else just turns into flamewars.
I saw one user yesterday post 10 lengthy, detailed answers in an hour, in 3 different programming languages. But the Mods aren't allowed by SE to consider that (or pretty much anything) to be an indicator that it's AI-generated.
Yeah, lets us keep providing free labor to help train somebody else's models, improve somebody else's infrastructure so that they can even more effectively dominate.
Or maybe you would want to pay for a version of stack overflow that is only curated by employed people ?
But I guess you would lose much content.
So it's not symmetric
But more importantly banning AI generated content from Stack Overflow doesn’t solve the “problem” you’re describing.
Its an absurd equivalence. A human can never "dominate" no matter how much they learn.
Training for free a closed, for-profit private enterprize infrastructure that will eventually make your preferred free collaboration platform obsolete is like the ultimate of naivete (or vested interest).
Put more succinctly I'm making two points: 1) The free work is already being done. 2) StackOverflow has been crawl-able since forever.
You'd have to address both of those problems before the concern that you're raising becomes relevant. Banning the posting of new generated content doesn't address either of those things.
The baseline you mention is already an issue that is being debated hotly.
My comment was that humans reviewing and fixing new AI content is exactly not what should be done in the current circumstances.
Somewhere an autocratic leader is laughing at you.
But spamming thousands of answers an hour automatically and wanting the community to do all the work is just not sustainable I feel. It'll also kill the sense of community if half the actors are bots.
answers are not either good or not. it's not binary. there is a lot of nuance and sometimes the questions and followups contain details that can take the answers in a completely new direction.
just telling if an answer is good or not is a lot of work. sometimes it requires and expert to figure it out. when the answers are given by a human and it's a good faith effort both people in the loop benefit from it. when the BS generation is automated there is zero incentive for a human being to even look and correct the answers. what is the incentive to do so?
You hit a good point here. If users can't be bothered to spend time, effort, energy and cognition into answering a question, why should the readers and correctors do so?
There are just so many interesting questions by having chatbots interact with StackOverflow:
What happens when AI-powered accounts are allowed to vote?
What kind of questions might AI-powered accounts be asking or be interested in?
How will the questions/answers/responses change between model revisions?
You can imagine things spiraling out of control. Perhaps there needs to be a chatbot version of StackOverflow to service all the questions that interest chatbots. :P
In practice this isn't possible. Lots of accepted answers are bad. Often for subtle reasons! An answer with a SQL injection vulnerability might get plenty of upvotes and be accepted, but it's objectively bad (even if it answers the question).
The problem is that there is no AI that's accurately fixing answers. AI only generates mediocre answers, it doesn't have the capacity to moderate mediocre answers. Humans simply can't keep up, or don't have the acumen to pick up on the subtle inaccuracies in accepted-but-bad answers (after all, that's often why they're looking for the answer).
Even with protections like rate limiting, I'm not sure you could prevent the majority of the damage that crappy AI can accomplish. Simply paying (pennies) for proxy servers with residential IPs gets around much of that, anyway.
My guess is the spam detection arms race will evolve into a content moderation arms race and we will end up with advanced AIs filtering and moderating content for us, to varying degrees of success and things like email, forums, and Q&A sites will become increasingly hands off.
Also StackOverflow is kind of gamified (a rare case of it working IMHO) and the rules of the game don't work so well when kind of good looking content is easy to generate. SE answers are hard to verify but easy to write. If writing becomes too easy, it is a recipe for spam - as has indeed happened.
It doesn't matter if they're generated by a 13-year-old in their bedroom, someone studying CS at university, a well-respected IC at a top tech company... or an AI.
If answers are good, keep them. If they're bad, downvote them. If they're redundant or off-topic or gibberish, delete them.
And to those asking why you would ever want AI-generated content on StackOverflow when you could just go to ChatGPT/Copilot/etc... it's because of all of the commentary. People are giving different answers, arguing the pros and cons, pointing out errors... all of the discussion around a StackOverflow question is usually just as valuable as any given answer, if not more so.
Of course, I understand that AI-powered accounts that can ask/answer hundreds of questions a minute are a problem simply because moderation can't keep up with them. But StackOverflow already has a lot of limits just for human accounts -- lots of actions you can't take until you've contributed certain amounts of value. Just extend these protections to do things like rate-limiting and so forth, to ensure that the normal ratio of content submitted vs. moderated stays constant and manageable.