News for hackers. Stuff that matters.
Re:Self-Adapting Language Models by kadushka on Friday June 13, 2025 @06:05PM (#44272721)
The most obvious blocker is catastrophic forgetting.
Re:Self-Adapting Language Models by solarwindy on Friday June 13, 2025 @08:26PM (#44273380)
Is that necessarily a blocker? As others in this thread have pointed out, this probably becomes possible only once sufficient compute is available for some form of non-public retraining, at the individual user level. In that case (and hand-waving away just how far off that is), does a model need to retain its generality?

Hypothetically (and perhaps more plausibly), a continually learning model that adapts to the context of a particular org / company / codebase / etc., could even be desirable.

Re:Self-Adapting Language Models by kadushka on Friday June 13, 2025 @11:43PM (#44274042)
Retraining the whole model from scratch every time you wanted it to learn something is not a solution.

does a model need to retain its generality?

Only if you want it to remain smart.