>Additionally, the training loop (which you call 'dumb') is just...a teacher-forced version of inference, which you call 'smart'?
What powers In context learning is not very well understood but it doesn't appear to be or really work exactly like just a non teacher-forced version of training. There are qualitative differences. The same models have no problem with this 'curse' when the information is provided in context for example.
>It's better to simply minimize the log-likelihood in a scalable way. Hand-engineered solutions rarely survive compared to strong-scaling ones.
I never said anything about it being bad.
I mean, yes. One distills information from a training set into a compressed representation, and the other generates a compressed representation that yields (more or less) fixed state space attractors. It's just inducing a bias over the state space of the network, nothing incredibly special, though I'd consider the initial stage of training to be the most important, as it is responsible for all of the ingest of all of the embedded information the network will be using during autoregressive inference (especially w.r.t. the context of doing so during longer generations).
So the notion of 'in context learning' is one I find to be a bit of an illusion, of course, as no actual learning is being done, just the induction of biases, which appears to give rise to a transiently-'better trained' network.
You could see this as a bit straightforward perhaps, but I feel it needs to be said.
It seems like you're referring to an associative Hebbian/Hopfield-like lookup, which the current 'dumb' optimizers already do. Better yet, the learning rates are normalized by the diagonal of the empirical Fisher so that the learning w.r.t. to some estimated expected information is more constant, for said associative lookup operation.
Additionally, the training loop (which you call 'dumb') is just...a teacher-forced version of inference, which you call 'smart'?
It's better to simply minimize the log-likelihood in a scalable way. Hand-engineered solutions rarely survive compared to strong-scaling ones.