Re:The Collapse of GPT
by NitpickLawyer on Saturday May 17, 2025 @02:00AM
(#44012329)
> This whole line of thought is sort of funny.
This line of thought was exacerbated by that one paper that was then parroted (hah!) by every influencer / negativist in the space. It didn't matter that the paper was badly executed, their setup was flawed and that it got rendered moot by the existence of LLama3 models. People still quote that, or the "articles" stemming from it.
That doesn't mean there aren't ways to train a model incorporating synthetic data without seeing model collapse