Only they know what's in their dataset. They have said previously however that they spend an immense amount of effort on curating a high quality dataset, so they probably are wary of this.

They are most certainly being fed LLM content. However, I think this "model collapse" narrative is over-subscribed. Here are some things to keep in mind:

(1) Real content is not generated via a synthetic loop: Humans use generative AI in complex ways, intermixing human-generated and AI-generated content. Imagine a person who writes the first draft of an essay, then uses ChatGPT to rewrite parts of it. These are certainly many human additions, modifications, and stylistic flourishes.

(2) The most dramatic effects of model collapse were seen when training multiple generations of AI agents on content generated by the previous agent. This is a very academic scenario.

(3) There is already a lot of junk consumed by these models. RLHF is aimed at eliminating these junk responses. I am not aware of any research that explores how the full training cycle is affected when RLHF is employed.

Also, there is a lot of training material out there that was not used by the original GPT-3 model. The primary limitation is hardware.

I have come across an increasing number of obviously generated content. Recipes, product reviews, and anything Buzzfeed was known for. I only expect more and more of it. Just wait until 2024's "top 38 React server component state management libraries you need to learn this year" posts come up on dev.to.

Edit: well look at that. I'm not saying this was generated, but it might as well could be. These "learn from these repos" posts are everywhere now.

https://dev.to/triggerdotdev/17-javascript-repositories-to-b...

It's also fairly well established now, I believe, that part of OpenAI's secret sauce is focusing on high-quality data sources; that is, probably those least likely to include unmodified ChatGPT outputs.

> This is a very academic scenario.

Is it going to remain academic? I can easily imagine the spammy content farm / listicle business model evolving to be fully automated, creating an input loop.

Sure, there will be some pollution. It's very multivariate and depends on factors like content split, generation quality, and novel information. A scenario in which all of your data is generated by the previous model and you run n training loops is academic.

It's also worth noting that when OpenAI created Whisper, they had to heuristically remove many transcripts from poor ASR systems, and they definitely didn't catch them all.

How would you positively identify LLM-written text? There's definitely a 'style' that's recognizable as "sounds like ChatGPT" but short of specific cliches (apparently for a while there were a lot of Amazon listings containing the text "As a large language model...")

If you're OpenAI you're going to look at all the training data which is available from people having used your LLM. If you ask it a question, it answers and you then ask it 20 more times to refine it then that's a massive amount of quality data for OpenAI. And the whole world is basically doing that, so they're going to have a massive amount of people who are asking the same questions and going through the same refinement processes to qualify their answers. This is going to lead to a good refinement of how those questions need to be answered going forward when it's compared to the datamodeling they already have.

It's going to be hard for competitors of course, but that's a different story.

Either by encoding a morse code into output by occasionally selecting suboptimal candidates, or by using a second AI trained to recognize AI. Neither method is known to work well.

If I were OpenAI I’d look in my logs.

I'm guessing that training an LLM to recognize LLM output should be an easy task.

If it's so easy, you could stand to make a lot of money bringing a quality product to early market.

Easy for those who already know how to train an LLM, I mean.