The Curse of Recursion: Training on Generated Data Makes Models Forget

Re:The Curse of Recursion: Training on Generated Data Makes ... by alterom on Saturday December 07, 2024 @01:44AM (#42347752)

The dignified way to describe the problem at hand is alluding to Brouwer's fixed-point theorem[1], with white noise as the fixed point.

The more practical way is alluding to The Human Centipede[2].

Either way, the feed-back loop doesn't result in a good output.

[1] https://en.wikipedia.org/wiki/Brouwer_fixed-point_theorem

[2] https://en.wikipedia.org/wiki/The_Human_Centipede_(First_Seq...

[ Parent | on HN ]

Re:The Curse of Recursion: Training on Generated Data Makes ... by benchmarkist on Saturday December 07, 2024 @02:18AM (#42347900)

Also the data processing inequality: https://en.wikipedia.org/wiki/Data_processing_inequality?use...

[ Parent | on HN ]

Re:The Curse of Recursion: Training on Generated Data Makes ... by cma on Saturday December 07, 2024 @03:01AM (#42348082)

And yet I prefer now to early big bang era of the universe, though technically reversible.

[ Parent | on HN ]

Re:The Curse of Recursion: Training on Generated Data Makes ... by benchmarkist on Saturday December 07, 2024 @03:10AM (#42348124)

The universe is not a Markov chain, in fact, no one knows what it is but locally we do know that entropy increases and the inevitable endpoint in our corner of the universe is complete annihilation. Your preferences are completely irrelevant in the local scheme of things.

[ Parent | on HN ]