News for hackers. Stuff that matters.

A comment on ARC-AGI-3. Parent comment

Re:ARC-AGI-3 by pants2 on Wednesday March 25, 2026 @06:14PM (#47523996)
The measurement metric is in-game steps. Unlimited reasoning between steps is fine.

This makes sense to me. Most actions have some cost associated, and as another poster stated it's not interesting to let models brute-force a solution with millions of steps.

[ Parent | on HN ]
Re:ARC-AGI-3 by famouswaffles on Wednesday March 25, 2026 @06:24PM (#47524084)
Same thing in this case. No Utility and just as arbitrary. None of the issues with the score change.

Models do not brute force solutions in that manner. If they did, we'd wait the lifetimes of several universes before we could expect a significant result.

Regardless, since there's a x5 step cuttof, 'brute forcing with millions of steps' was never on the table.

[ Parent | on HN ]