News for hackers. Stuff that matters.
Re:ARC-AGI without pretraining by jshmrsn on Tuesday March 04, 2025 @04:50PM (#43260154)
If the machine can decide how to train itself (adjust weights) when faced with a type of problem it hasn’t seen before, then I don’t think that would go against the spirit of general intelligence. I think that’s basically what humans do when they decide to get better at something, they figure out how to practice that task until they get better at it.
Re:ARC-AGI without pretraining by pona-a on Tuesday March 04, 2025 @05:10PM (#43260336)
In-context learning is a very different problem from regular prediction. It is quite simple to fit a stationary solution to noisy data, that's just a matter of tuning some parameters with fairly even gradients. In-context learning implies you're essentially learning a mesa-optimizer for the class of problems you're facing, which in the form of transformers means essentially means fitting something not that far from a differentiable Turing machine with no inductive biases.