News for hackers. Stuff that matters.
Re:The Collapse of GPT by wrsh07 on Friday May 16, 2025 @09:14PM (#44011275)
This whole line of thought is sort of funny. Yes you can try training a model on synthetic data in such a way that it experiences model collapse

That doesn't mean there aren't ways to train a model incorporating synthetic data without seeing model collapse

Re:The Collapse of GPT by NitpickLawyer on Saturday May 17, 2025 @02:00AM (#44012329)
> This whole line of thought is sort of funny.

This line of thought was exacerbated by that one paper that was then parroted (hah!) by every influencer / negativist in the space. It didn't matter that the paper was badly executed, their setup was flawed and that it got rendered moot by the existence of LLama3 models. People still quote that, or the "articles" stemming from it.