News for hackers. Stuff that matters.

A comment on The Collapse of GPT. Parent comment

Re:The Collapse of GPT by ComplexSystems on Saturday May 17, 2025 @09:57AM (#44014376)
So does human content. Much of the original data that GPT was trained on is Reddit posts and web pages on the internet - stuff that isn't exactly known for being a source of high quality facts.
[ Parent | on HN ]
Re:The Collapse of GPT by lelanthran on Saturday May 17, 2025 @12:48PM (#44015513)
That's still very different to "drift error compounding".

If you output a mere 5% drift error and then use that as input you only need a few cycles (single digits) before your output is more erroneous than correct.

We are already partly into the second cycle. By the fifth the LLM would be mostly useless.

[ Parent | on HN ]