News for hackers. Stuff that matters.
Re:The Collapse of GPT by ninetyninenine on Friday May 16, 2025 @11:38PM (#44011863)
I mean imagine Linear least squares on a 2D graph.

I have a best fit line. Then I take random data on that line to train a new line.

I pretty much get the same line.

From an intuitive perspective... it doesn't get worse. At worst it stays the same.

Now imagine something a bit more complex. I have a best fit curve that's very close to a line.

I use random data from that curve to train a new best fit line.

I get something different now. Not necessarily worse.

I mean literally just take all your ideas of ML and just imagine it on the 2D plane doing curve fitting. If retraining new lines from generated data doesn't necessarily make things worse.

There are no comments on this comment yet.