Please learn a bit of combinatorics.
> After all, the human solving these problem doesn't make 10k attempts before getting a solution, do they?
No. People have much better "early rejection", also human brain has massive parallel compute capacity.
It's ridiculous to demand GPT-4 performs as good as a human. Obviously its vision is much worse and it doesn't have 'video' and physics priors people have, so it has to guess more times.
Brute searching literally means generating solutions until one works. Which is exactly what is being done here.
> Please learn a bit of combinatorics.
Don't be condescending - I understand the problem space just fine. Fine enough to realise that the problem was constructed specifically to ensure that "solutions" such as this just won't work.
Which is why this "solution" is straight-up broken (doesn't meet the target, exceeds the computationally bounds, etc).
> It's ridiculous to demand GPT-4 performs as good as a human.
Wasn't the whole point of this prize to spur interest in a new approach to learning? What does GPT-[1234] have to do with the contest rules? Especially since this solution broke those rules anyway?
> Obviously its vision is much worse and it doesn't have 'video' and physics priors people have, so it has to guess more times.
That's precisely my point - it has to guess. Humans aren't guessing for those types of problems (not for the few that I saw anyway).
It's similar that a lot of wrong answers are being thrown up, but I think this is more like a probabilistic system which is being pruned than a walk of the solution space. It's much smarter, but not as smart as we would like.
Sure, but not an exhaustive one - you stop when you get a solution[1]. Brute force does not require an exhaustive search in order to be called brute-force.
GP was using the argument that because it is not exhaustive, it cannot be brute-force. That's the wrong argument. Brute-force doesn't have to be exhaustive to be brute-force.
[1] Or a good enough solution.
You're 100% WRONG on everything you wrote. You don't get change established terminology just to denigrate the approach you don't like.
It seems you are just biased against LLMs. But this test is as much a test for limits of LLM abilities as it is a quest for a new approach. It's open ended.
And it's not like the author of the article claims he has AGI - he just shows the limits of LLM-based solutions.
Who hasn't?
> You're 100% WRONG on everything you wrote.
Maybe you should update the wikipedia page, then all the other textbooks, that uses a definition of brute-force that matches my understanding of it.
From https://en.wikipedia.org/wiki/Brute-force_search
> Therefore, brute-force search is typically used when the problem size is limited, or when there are problem-specific heuristics that can be used to reduce the set of candidate solutions to a manageable size.
Further, in the same page https://en.wikipedia.org/wiki/Brute-force_search#Speeding_up...
> One way to speed up a brute-force algorithm is to reduce the search space, that is, the set of candidate solutions, by using heuristics specific to the problem class.
I mean, the approach under discussion is literally exactly this.
Now, Mr "ACM ICPC, studied algorithms for years", where's your reference that reducing the solution space using heuristics results in a non-brute-force algorithm?
For a reference, check Cormen's "Introduction to Algorithms". Every mention of brute-force search is specifically to exhaustive search is which not feasible for bigger spaces.
> I mean, the approach under discussion is literally exactly this.
It's literally not. It DOES NOT REDUCE the candidate set. It generates most likely candidates, but it doesn't reduce anything.
You lack basic understanding. Solutions are pixels grids, not Python programs. There's no search over pixel grids in the article. Not every search is exhaustive search.
This is like saying theoretical physicists are "brute-forcing" physics by generating candidate theories and testing them. Ridiculous.
Also: lol at your "who hasn't" comment. Because you clearly haven't.
Reference? Link, even?
> don't care about whatever random wiki page you might find to "support your claims".
That isn't some "random wiki" page; that's the wikipedia page for this specific term.
I'm not claiming to have defined this term, I'm literally saying I only agree with the sources for this term.
> Also: lol at your "who hasn't" comment. Because you clearly haven't.
Talk about cringe-worthy.
Sure, here's definition for "brute force" from university textbook material written by pllk, who has taught algorithms for 20 years and holds a 2400 rating on Codeforces:
https://tira.mooc.fi/kevat-2024/osa9/
"Yleispätevä tapa ratkaista hakuongelmia on toteuttaa raakaan voimaan (brute force) perustuva haku, joka käy läpi kaikki ratkaisut yksi kerrallaan."
edit:
Here's an English language book written by the same author, though the English source does not precisely define the term:
In chapter 5:
"Complete search is a general method that can be used to solve almost any algorithm problem. The idea is to generate all possible solutions to the problem using brute force ..."
And a bit further down chapter 5:
"We can often optimize backtracking by pruning the search tree. The idea is to add ”intelligence” to the algorithm so that it will notice as soon as possible if a partial solution cannot be extended to a complete solution. Such optimizations can have a tremendous effect on the efficiency of the search."
Your mistake is that you for some reason believe that any search over solution space is a brute force solution. But there are many ways to search over a solution space. A "dumb search" over solution space is generally considered to be brute force, whereas a "smart search" is generally not considered to be brute force.
Here's the Codeforces profile of the author: https://codeforces.com/profile/pllk
edit 2:
Ok now I think I understand what causes your confusion. When an author writes "One way to speed up a brute-force algorithm ..." you think that the algorithm can still be called "brute force" after whatever optimizations were applied. No. That's not what that text means. This is like saying "One way to make a gray car more colorful is by painting it red". Is it still a gray car after it has been painted red? No it is not.
I mean, generating tens of thousands of possible solutions, to find one that works does not, to me, signify AGI.
After all, the human solving these problem doesn't make 10k attempts before getting a solution, do they?
The approach here, due to brute force, can't really scale: if a random solution to a very simple problem has a 1/10k chance of being right, you can't scale this up to non-trivial problems without exponentially increasing the computational power used. Hence, I feel this is brute-force.