Re:Self-Adapting Language Models
by johnsmith1840 on Sunday June 15, 2025 @01:56PM
(#44283836)
Retrain from stratch does technically solve it.
But it doesn't solve the time aspect.
You need to randomize data in order to train to best quality. In doing that the model has no idea t0 was before t1000. If you don't you get model collapse or heavy bias.
But it doesn't solve the time aspect.
You need to randomize data in order to train to best quality. In doing that the model has no idea t0 was before t1000. If you don't you get model collapse or heavy bias.
Some attempts at it but nothing crazy effective.