News for hackers. Stuff that matters.

A comment on The Curse of Recursion: Training on Generated Data Makes Models Forget. Parent comment

Re:The Curse of Recursion: Training on Generated Data Makes ... by DennisP on Tuesday June 13, 2023 @10:01PM (#36320450)
The trick will be figuring out what part of your training data was actually made by humans.
[ Parent | on HN ]
Re:The Curse of Recursion: Training on Generated Data Makes ... by castis on Tuesday June 13, 2023 @10:27PM (#36320666)
Run it through an LLM and see if it gets the disease?
[ Parent | on HN ]
Re:The Curse of Recursion: Training on Generated Data Makes ... by DennisP on Wednesday June 14, 2023 @09:11AM (#36325371)
But a single document doesn't break the LLM. The problem happens when lots of training documents were AI-generated.
[ Parent | on HN ]
Re:The Curse of Recursion: Training on Generated Data Makes ... by flangola7 on Wednesday June 14, 2023 @09:48AM (#36325883)
> The trick will be figuring out what part of your training data was actually made by humans.

My point was that this will not be necessary for the same reason it isn't necessary to filter out human-made content.

[ Parent | on HN ]