News for hackers. Stuff that matters.
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by modeless on Friday December 20, 2024 @01:48PM (#42473688)
Solving ARC doesn't mean we have AGI. Also o3 presumably isn't doing program synthesis, seemingly proving Francois wrong on that front. (Not sure I believe the speculation about o3's internals in the link.)

What I'm saying is the fact that as models are getting better at reasoning they are also scoring better on ARC proves that it is measuring something relating to reasoning. And nobody else has come up with a comparable benchmark that is so easy for humans and so hard for LLMs. Even today, let alone five years ago when ARC was released. ARC was visionary.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by hdjjhhvvhga on Friday December 20, 2024 @02:00PM (#42473808)
Your argumentation seems convincing but I'd like to offer a competitive narrative: any benchmark that is public becomes completely useless because companies optimize for it - especially AI that depends on piles of money and they need some proof they are developing.

That's why I have some private benchmarks and I'm sorry to say that the transition from GTP4 to o1 wasn't unambiguously a step forward (in some tasks yes, in some not).

On the other hand, private benchmarks are even less useful to the general public than the public ones, so we have to deal with what we have - but many of us just treat it as noise and don't give it much significance. Ultimately, the models should defend themselves by performing the tasks individual users want them to do.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by stonemetal12 on Friday December 20, 2024 @02:15PM (#42473961)
Rather any Logic puzzle you post on the internet as something AIs are bad at is in the next round of training data so AIs get better at that specific question. Not because AI companies are optimizing for a benchmark but because they suck up everything.
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by modeless on Friday December 20, 2024 @02:24PM (#42474071)
ARC has two test sets that are not posted on the Internet. One is kept completely private and never shared. It is used when testing open source models and the models are run locally with no internet access. The other test set is used when testing closed source models that are only available as APIs. So it could be leaked in theory, but it is still not posted on the internet and can't be in any web crawls.

You could argue that the models can get an advantage by looking at the training set which is on the internet. But all of the tasks are unique and generalizing from the training set to the test set is the whole point of the benchmark. So it's not a serious objection.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by foobiekr on Friday December 20, 2024 @07:14PM (#42476419)
Given the delivery mechanism for OpenAI, how do they actually keep it private?
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by modeless on Friday December 20, 2024 @07:51PM (#42476607)
> So it could be leaked in theory

That's why they have two test sets. But OpenAI has legally committed to not training on data passed to the API. I don't believe OpenAI would burn their reputation and risk legal action just to cheat on ARC. And what they've reported is not implausible IMO.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by sensanaty on Saturday December 21, 2024 @07:20AM (#42479262)
Yeah I'm sure the Microsoft-backed company headed by Mr. Worldcoin Altman whose sole mission statement so far has been to overhype every single product they released wouldn't dare cheat on one of these benchmarks that "prove" AGI (as they've been claiming since GPT-2).
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by QuantumGood on Friday December 20, 2024 @02:04PM (#42473847)
Gaming the benchmarks usually needs to be considered first when evaluating new results.
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by bubblyworld on Friday December 20, 2024 @02:23PM (#42474053)
I think gaming the benchmarks is encouraged in the ARC AGI context. If you look at the public test cases you'll see they test a ton of pretty abstract concepts - space, colour, basic laws of physics like gravity/magnetism, movement, identity and lots of other stuff (highly recommend exploring them). Getting an AI to do well at all, regardless of whether it was gamed or not, is the whole challenge!
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by chaps on Friday December 20, 2024 @02:14PM (#42473949)
Honestly, is gaming benchmarks actually a problem in this space in that it still shows something useful? Just means we need more benchmarks, yeah? It really feels not unlike keggle competitions.

We do the same exact stuff with real people with programming challenges and such where people just study common interview questions rather than learning the material holistically. And since we know that people game these interview type questions, we can adjust the interview processes to minimize gamification.... which itself leads to gamification and back to step one. That's not ideal an ideal feedback loop of course, but people still get jobs and churn out "productive work" out of it.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by ben_w on Friday December 20, 2024 @02:19PM (#42474013)
AI are very good at gaming benchmarks. Both as overfitting and as Goodhart's law, gaming benchmarks has been a core problem during training for as long as I've been interested in the field.

Sometimes this manifests as "outside the box thinking", like how a genetic algorithm got an "oscillator" which was really just an antenna.

It is a hard problem, and yes we still both need and can make more and better benchmarks; but it's still a problem because it means the benchmarks we do have are overstating competence.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by chaps on Friday December 20, 2024 @02:54PM (#42474378)
We're in agreement!

What's endlessly interesting to me with all of this is how surprisingly quick the benchmarking feedback loops have become plus the level of scrutiny each one receives. We (as a culture/society/whatever) don't really treat human benchmarking criteria with the same scrutiny such that feedback loops are useful and lead to productive changes to the benchmarking system itself. So from that POV it feels like substantial progress continues to be made through these benchmarks.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by CamperBob2 on Friday December 20, 2024 @02:38PM (#42474217)
The idea behind this particular benchmark, at least, is that it can't be gamed. What are some ways to game ARC-AGI, meaning to pass it without developing the required internal model and insights?

In principle you can't optimize specifically for ARC-AGI, train against it, or overfit to it, because only a few of the puzzles are publicly disclosed.

Whether it lives up to that goal, I don't know, but their approach sounded good when I first heard about it.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by psb217 on Friday December 20, 2024 @03:25PM (#42474674)
Well, with billions in funding you could task a hundred or so very well paid researchers to do their best at reverse engineering the general thought process which went into ARC-AGI, and then generate fresh training data and labeled CoTs until the numbers go up.
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by CamperBob2 on Friday December 20, 2024 @03:29PM (#42474722)
Right, but the ARC-AGI people would counter by saying they're welcome to do just that. In doing so -- again in their view -- the researchers would create a model that could be considered capable of AGI.

I spent a couple of hours looking at the publicly-available puzzles, and was really impressed at how much room for creativity the format provides. Supposedly the puzzles are "easy for humans," but some of them were not... at least not for me.

(It did occur to me that a better test of AGI might be the ability to generate new, innovative ARC-AGI puzzles.)

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by psb217 on Friday December 20, 2024 @07:37PM (#42476547)
It's tricky to judge the difficulty of these sorts of things. Eg, breadth of possibilities isn't an automatic sign of difficulty. I imagine the space of programming problems permits as much variety as ARC-AGI, but since we're more familiar with problems presented as natural language descriptions of programming tasks, and since we know there's tons of relevant text on the web, we see the abstract pictographic ARC-AGI tasks as more novel, challenging, etc. But, to an LLM, any task we can conceive of will be (roughly) as familiar as the amount of relevant training data it's seen. It's legitimately hard to internalize this.

For a space of tasks which are well-suited to programmatic generation, as ARC-AGI is by design, if we can do a decent job of reverse engineering the underlying problem generating grammar, then we can make an LLM as familiar with the task as we're willing to spend on compute.

To be clear, I'm not saying solving these sorts of tasks is unimpressive. I'm saying that I find it unsuprising (in light of past results) and not that strong of a signal about further progress towards the singularity, or FOOM, or whatever. For any of these closed-ish domain tasks, I feel a bit like they're solving Go for the umpteenth time. We now know that if you collect enough relevant training data and train a big enough model with enough GPUs, the training loss will go down and you'll probably get solid performance on the test set. Trillions of reasonably diverse training tokens buys you a lot of generalization. Ie, supervised learning works. This is the horse Ilya Sutskever's ridden to many glorious victories and the big driver of OpenAI's success -- a firm belief that other folks were leaving A LOT of performance on the table due to a lack of belief in the power of their own inventions.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by refulgentis on Friday December 20, 2024 @02:34PM (#42474180)
> Solving ARC doesn't mean we have AGI. Also o3 presumably isn't doing program synthesis, seemingly proving Francois wrong on that front.

Agreed.

> And nobody else has come up with a comparable benchmark that is so easy for humans and so hard for LLMs.

? There's plenty.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by modeless on Friday December 20, 2024 @02:50PM (#42474336)
I'd love to hear about more. Which ones are you thinking of?
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by refulgentis on Friday December 20, 2024 @03:15PM (#42474585)
- "Are You Human" https://arxiv.org/pdf/2410.09569 is designed to be directly on target, i.e. cross cutting set of questions that are easy for humans, but challenging for LLMs, Instead of one type of visual puzzle. Much better than ARC for the purpose you're looking for.

- SimpleBench https://simple-bench.com/ (similar to above; great landing page w/scores that show human / ai gap)

- PIQA (physical question answering, i.e. "how do i get a yolk out of a water bottle", common favorite of local llm enthusiasts in /r/localllama https://paperswithcode.com/dataset/piqa

- Berkeley Function-Calling (I prefer https://gorilla.cs.berkeley.edu/leaderboard.html)

AI search googled "llm benchmarks challenging for ai easy for humans", and "language model benchmarks that humans excel at but ai struggles with", and "tasks that are easy for humans but difficult for natural language ai".

It also mentioned Moravec's Paradox is a known framing of this concept, started going down that rabbit hole because the resources were fascinating, but, had to hold back and submit this reply first. :)

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by modeless on Friday December 20, 2024 @04:26PM (#42475208)
Thanks for the pointers! I hadn't seen Are You Human. Looks like it's only two months old. Of course it is much easier to design a test specifically to thwart LLMs now that we have them. It seems to me that it is designed to exploit details of LLM structure like tokenizers (e.g. character counting tasks) rather than to provide any sort of general reasoning benchmark. As such it seems relatively straightforward to improve performance in ways that wouldn't necessarily represent progress in general reasoning. And today's LLMs are not nearly as far from human performance on the benchmark as they were on ARC for many years after it was released.

SimpleBench looks more interesting. Also less than two months old. It doesn't look as challenging for LLMs as ARC, since o1-preview and Sonnet 3.5 already got half of the human baseline score; they did much worse on ARC. But I like the direction!

PIQA is cool but not hard enough for LLMs.

I'm not sure Berkeley Function-Calling represents tasks that are "easy" for average humans. Maybe programmers could perform well on it. But I like ARC in part because the tasks do seem like they should be quite straightforward even for non-expert humans.

Moravec's paradox isn't a benchmark per se. I tend to believe that there is no real paradox and all we need is larger datasets to see the same scaling laws that we have for LLMs. I see good evidence in this direction: https://www.physicalintelligence.company/blog/pi0

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by refulgentis on Friday December 20, 2024 @06:03PM (#42475995)
> "I'm not sure Berkeley Function-Calling represents tasks that are easy for average humans. Maybe programmers could perform well on it."

Functions in this context are not programming function calls. In this context, function calls are a now-deprecated LLM API name for "parse input into this JSON template." No programmer experience needed. Entity extraction by another name, except, that'd be harder: here, you're told up front exactly the set of entities to identify. :)

> "Moravec's paradox isn't a benchmark per se."

Yup! It's a paradox :)

> "Of course it is much easier to design a test specifically to thwart LLMs now that we have them"

Yes.

Though, I'm concerned a simple yes might be insufficient for illumination here.

It is a tautology (it's easier to design a test that $X fails when you have access to $X), and it's unlikely you meant to just share a tautology.

A potential unstated-but-maybe-intended-communication is "it was hard to come up with ARC before LLMs existed" --- LLMs existed in 2019 :)

If they didn't, a hacky way to come up with a test that's hard for the top AIs at the time, BERT-era, would be to use one type of visual puzzle.

If, for conversations sake, we ignore that it is exactly one type of visual puzzle, and that it wasn't designed to be easy for humans, then we can engage with: "its the only one thats easy for humans, but hard for LLMs" --- this was demonstrated as untrue as well.

I don't think I have much to contribute past that, once we're at "It is a singular example of a benchmark thats easy for humans but nigh-impossible for llms, at least in 2019, and this required singular insight", there's just too much that's not even wrong, in the Pauli sense, and it's in a different universe from the original claims:

- "Congratulations to Francois Chollet on making the most interesting and challenging LLM benchmark so far."

- "A lot of people have criticized ARC as not being relevant or indicative of true reasoning...The fact that [o-series models show progress on ARC proves that what it measures really is relevant and important for reasoning."

- "...nobody could quantify exactly the ways the models were deficient..."

- "What we need right now are "easy" benchmarks that these models nevertheless fail."

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by CamperBob2 on Friday December 20, 2024 @04:58PM (#42475474)
How long has SimpleBench been posted? Out of the first 6 questions at https://simple-bench.com/try-yourself, o1-pro got 5/6 right.

It was interesting to see how it failed on question 6: https://chatgpt.com/c/6765e70e-44b0-800b-97bd-928919f04fbe

Apparently LLMs do not consider global thermonuclear war to be all that big a deal, for better or worse.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by Pannoniae on Friday December 20, 2024 @05:16PM (#42475619)
Don't worry, I also got that wrong :) I thought her affair would be the biggest problem for John.
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by jquery on Friday December 20, 2024 @06:04PM (#42476000)
John was an ex, not her partner. Tricky.
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by HarHarVeryFunny on Friday December 20, 2024 @07:42PM (#42476568)
> o3 presumably isn't doing program synthesis

I'd guess it's doing natural language procedural synthesis, the same way a human might (i.e. figuring the sequence of steps to effect the transformation), but it may well be doing (sub-)solution verification by using the procedural description to generate code whose output can then be compared to the provided examples.

While OpenAI haven't said exactly what the architecture of o1/o3 are, the gist of it is pretty clear - basically adding "tree" search and iteration on top of the underlying LLM, driven by some RL-based post-training that imparts generic problem solving biases to the model. Maybe there is a separate model orchestrating the search and solution evaluation.

I think there are many tasks that are easy enough for humans but hard/impossible for these models - the ultimate one in terms of commercial value would be to take an "off the shelf model" and treat it as an intern/apprentice and teach it to become competent in a entire job it was never trained on. Have it participate in team meetings and communications, and become a drop-in replacement for a human performing that job (any job that an be performed remotely without a physical presence).