AI models collapse when trained on recursively generated data

Re:AI models collapse when trained on recursively generated ... by abeppu on Wednesday July 24, 2024 @12:53PM (#41059047)

Yes -- said another way, if you're an ML researcher and you have human-provided (scraped) data, and an ability to generate synthetic data, then until recently, you had a controllable parameter: how much of your training data for your new model should be synthetic? You can vary this, run multiple experiments, and choose how much synthetic data to use -- and you can vary the specific configs about how that synthetic data is generated.

If synthetic data is mixed into your upstream data sources in a way you cannot control, then your ML team loses a valuable controllable parameter.

[ Parent | on HN ]

Re:AI models collapse when trained on recursively generated ... by miki123211 on Wednesday July 24, 2024 @01:25PM (#41059467)

You still have some that control, but in a much more indirect way.

There are three kinds of data now, synthetic, pre-2022 and current. Everything pre-2022 is definitely written by humans, synthetic data is still synthetic, and post-2022 is a mix of both.

I wouldn't be surprised if "AI detectors" work somewhat for this use case. They're biased, far from accurate and a terrible idea if you need to make important decisions (like whether to expel a student for cheating), but there's quite a large room for errors here.

[ Parent | on HN ]

Re:AI models collapse when trained on recursively generated ... by uberswe on Wednesday July 24, 2024 @01:51PM (#41059735)

> Everything pre-2022 is definitely written by humans

I'm not sure if methods like article spinning counts as written by humans. This is something you could automate before AI and it would take a human written article and randomly swap words with similar meaning throughout to make it seem original.

[ Parent | on HN ]

Re:AI models collapse when trained on recursively generated ... by derefr on Wednesday July 24, 2024 @05:04PM (#41062097)

Don’t forget machine-translated texts, where until ~2017 the translation was likely done by something much dumber / semantically lossy than an LLM, and after 2017 was basically done by an early form of LLM (the Transformers architecture originating in Google Translate.)

Many historical English-language news reports published on the English-language websites of foreign news media from non-English-speaking countries, from 1998 (Babelfish era) to ~a few months ago, may be unreliable training data for this reason.

[ Parent | on HN ]

Re:AI models collapse when trained on recursively generated ... by mrbungie on Wednesday July 24, 2024 @06:35PM (#41062936)

They do work detecting LLM outputs that are sampled "naively" (when the model/user is really not trying to pass it as human output).

I copied a prompt translated from spanish to english using ChatGPT Plus in a GPT-4o Azure OpenAI Service endpoint. It did work in Spanish but didn't run in english because the default AOS Content Filters detected a jailbreak intent. It was quite weird.

[ Parent | on HN ]