News for hackers. Stuff that matters.
Re:ARC-AGI without pretraining by ta8645 on Tuesday March 04, 2025 @04:03PM (#43259790)
I'd say that it takes intelligence to encode knowledge, and the more knowledge you have, the more intelligently you can encode further knowledge, in a virtuous cycle. But once you have a data set of knowledge, there's nothing to emerge, there are no side effects. It just sits there doing nothing. The intelligence is in the algorithms that access that encoded knowledge to produce something else.
Re:ARC-AGI without pretraining by esafak on Tuesday March 04, 2025 @04:47PM (#43260128)
The data set is flawed, noisy, and its pieces are disconnected. It takes intelligence to correct its flaws and connect them parsimoniously.
Re:ARC-AGI without pretraining by ta8645 on Tuesday March 04, 2025 @05:06PM (#43260300)
It takes knowledge to even know they're flawed, noisy, and disconnected. There's no reason to "correct" anything, unless you have knowledge that applying previously "understood" data has in fact produced deficient results in some application.

That's reinforcement learning -- an algorithm, which requires accurate knowledge acquisition, to be effective.

Re:ARC-AGI without pretraining by esafak on Tuesday March 04, 2025 @05:58PM (#43260658)
Every statistical machine learning algorithm, including RL, deals with noisy data. The process of fitting aims to remove the sampling noise, revealing the population distribution, thereby compressing it into a model.

The argument being advanced is that intelligence is the proposal of more parsimonious models, aka compression.

Re:ARC-AGI without pretraining by ta8645 on Tuesday March 04, 2025 @06:16PM (#43260783)
I've lost track of what we're disagreeing about.