Self-Adapting Language Models

Re:Self-Adapting Language Models by mackenziebowes on Friday June 13, 2025 @09:36PM (#44273618)

I can't be certain, I'm not at all an AI engineer or math guy, but I think at the "wake up" point you equalize instances. Like during 'sleep' some list of functions/operations `m` are applied to model weights `n` producing a new model, `n + 1`. Wouldn't you just clone `n + 1`, send it to work, and start a new training run `m + 1` to make `n + 2`?

[ Parent | on HN ]

Re:Self-Adapting Language Models by notpushkin on Saturday June 14, 2025 @02:11AM (#44274560)

This was my first idea as well. Keep training continuously and redeploy clones after each cycle. From a layman perspective this seems reasonable :thinking:

[ Parent | on HN ]

Re:Self-Adapting Language Models by maleldil on Saturday June 14, 2025 @10:01PM (#44280071)

You can't realistically keep training the same model forever, or it will start forgetting things it knew before. The proper name for this is "catastrophic forgetting".

[ Parent | on HN ]