AI models collapse when trained on recursively generated data

Re:AI models collapse when trained on recursively generated ... by nick486 on Wednesday July 24, 2024 @01:28PM (#41059506)

the equivalent here would be a child learning from a textbook he has written himself.

not sure how effective that would be, if it was his only source of learning.

[ Parent | on HN ]

Re:AI models collapse when trained on recursively generated ... by throwup238 on Wednesday July 24, 2024 @01:52PM (#41059747)

Well that’s what the TFA is about. If you indiscriminately ingest synthetic data into training - the child learning from their own textbook - the model collapses.

The SOTA is to use a discriminator (often another LLM or ML algo) to select the best output before feeding it into the training data. That’s what OpenAI, Anthropic, et al have been doing. One of them just published a paper about it a few weeks ago.

[ Parent | on HN ]