Re:ARC-AGI without pretraining
by pona-a on Tuesday March 04, 2025 @05:10PM
(#43260336)
In-context learning is a very different problem from regular prediction. It is quite simple to fit a stationary solution to noisy data, that's just a matter of tuning some parameters with fairly even gradients. In-context learning implies you're essentially learning a mesa-optimizer for the class of problems you're facing, which in the form of transformers means essentially means fitting something not that far from a differentiable Turing machine with no inductive biases.
[ Parent
| on HN ]
There are no comments on this comment yet.