even if we bought this negative result as somehow “proving impossibility”, i’m not convinced plasticity is necessary for intelligence
huge respect for richard sutton though
More specifically: it is highly implausible that an AI system could learn to improve itself beyond human capability if it does not have long-term plasticity: how would it be able to reflect upon and extend its discoveries if it's not able to learn new things during its operation?
(That said, I agree plasticity is key to the most powerful systems. A human race with anterograde amnesia would have long ago gone extinct.)
If I'm a human tasked with editing video (which is the field my startup[0] is in) and a completely new video format comes in, I need the long term plasticity to learn how to use it so I can perform my work.
If a sufficiently intelligent version of our AI model is tasked with editing these videos, and a completely new video format comes in, it does not need to learn to handle it. Not if this model is smart enough to iterate a new model that can handle it.
The new skills and knowledge do not need to be encoded in "the self" when you are a bunch of bytes that can build your successor out of more bytes.
Or, in popular culture terms, the last 30 seconds of this Age of Ultron clip[1].
What do you think training (and fine-tuning) does?
No LLM currently adapts to the tasks its given with an iteration cycle shorter than on the order of months (assuming your conversations serve as future training data; otherwise not at all).
No current LLM can digest its "experiences", form hypotheses (at least outside of being queried), run thought experiments, then actual experiments, and then update based on the outcome.
Not because it's fundamentally impossible (it might or might not be), but because we practically haven't built anything even remotely approaching that type of architecture.
Modern ANN architectures are not actually capable of long-term learning in the same way animals are, even stodgy old dogs that don't learn new tricks. ANNs are not a plausible model for the brain, even if they emulate certain parts of the brain (the cerebellum, but not the cortex)
I will add that transformers are not capable of recursion, so it's impossible for them to realistically emulate a pigeon's brain. (you would need millions of layers that "unlink chains of thought" purely by exhaustion)