News for hackers. Stuff that matters.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by amluto on Wednesday September 17, 2025 @04:52PM (#45281281)
I've contemplated this a bit, and I think I have a bit of an unconventional take:

First, this is really impressive.

Second, with that out of the way, these models are not playing the same game as the human contestants, in at least two major regards. First, and quite obviously, they have massive amounts of compute power, which is kind of like giving a human team a week instead of five hours. But the models that are competing have absolutely massive memorization capacity, whereas the teams are allowed to bring a 25-page PDF with them and they need to manually transcribe anything from that PDF that they actually want to use in a submission.

I think that, if you gave me the ability to search the pre-contest Internet and a week to prepare my submissions, I would be kind of embarrassed if I didn't get gold, and I'd find the contest to be rather less interesting than I would find the real thing.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by asboans on Wednesday September 17, 2025 @06:08PM (#45281965)
Firstly, automobiles are really impressive.

Second, with that out the way, these cars are not playing the same game as horses… first, and quite obviously they have massive amounts of horsepower, which is kind of like giving a team of horses… many more horses. But also cars have an absolutely massive fuel capacity. Petrol is such an efficient store of chemical energy compared to hay and cars can store gallons of it.

I think if you give my horse the ability of 300 horses and fed it pure gasoline, I would be kind of embarrassed if it wasn’t able to win a horse race.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by furyofantares on Wednesday September 17, 2025 @08:10PM (#45283131)
Yeah man, and it would be wild to publish an article titled "Ford Mustang and Honda Civic win gold in the 100 meter dash at the Olympics" if what happened was the companies drove their cars 100 meters and tweeted that they did it faster than the Olympians had run.

Actually that's too generous, because the humans are given a time limit in ICPC, and there's no clear mapping to say how the LLM's compute should be limited to make a comparison.

It IS an interesting result to see how models can do on these tests - and it's also a garbage headline.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by LPisGood on Wednesday September 17, 2025 @08:22PM (#45283249)
I think your analogy is interesting but it falls apart because “moving fast” is not something we consider uniquely human, but “solving hard abstract problems” is
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by furyofantares on Wednesday September 17, 2025 @08:29PM (#45283299)
Not my analogy, parent is the one who brought up automobiles. Maybe that's who you meant to reply to.

I'm talking about the headline saying they "won gold" at a competition they didn't, and couldn't, compete in.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by in-silico on Wednesday September 17, 2025 @08:31PM (#45283311)
Cars going faster than humans or horses isn't very interesting these days, but it was 100+ years ago when cars were first coming on the scene.

We are at that point now with AI, so a more fitting headline analogy would be "In a world first, automobile finishes with gold-winning time in horse race".

Headlines like those were a sign that cars would eventually replace horses in most use-cases, so the fact that we could be in the the same place now with AI and humans is a big deal.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by gwbrooks on Wednesday September 17, 2025 @10:21PM (#45284336)
It was more than interesting 100+ years ago -- it was the subject of wildly inconsistent, often fear-based (or incumbent-industry-based) regulation.

A vetoed 1896 Pennsylvania law would have required drivers who encountered livestock to "disassemble the automobile" and "conceal the various components out of sight, behind nearby bushes until [the] equestrian or livestock is sufficiently pacified". The Locomotive on Highways Act of 1865 required early motorized vehicles to be preceded by a person on foot waving a red flag or carrying a red lantern and blowing a horn.

It might not quite look like that today, but wild-eyed, fear-based regulation as AI use grows is a real possibility. And at least some of it will likely seem just as silly in hindsight.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by jdietrich on Wednesday September 17, 2025 @11:12PM (#45284816)
For more than thirty years, the speed limit for cars in Britain was 4mph - a self-propelled vehicle travelling faster than walking pace was obviously unconscionably dangerous.

To celebrate the raising of the speed limit to a daring 12mph, a group of motorists organised a drive from London to Brighton. At the time, driving 54 miles in a single day was seen as an audacious feat and few people imagined that such a great distance could be travelled in such complicated and newfangled contraptions without mechanical incident.

For decades, the car was seen as a plaything for the wealthy that served no practical purpose. The car only became an important mode of transportation after very many false starts and against strong opposition.

https://en.wikipedia.org/wiki/Locomotive_Acts#Locomotives_Ac...

https://en.wikipedia.org/wiki/London_to_Brighton_Veteran_Car...

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by krisoft on Wednesday September 17, 2025 @08:46PM (#45283448)
> what happened was the companies drove their cars 100 meters and tweeted that they did it faster than the Olympians had run

That would be indeed an interesting race around the time cars were invented. Today that would be silly, since everyone knows what cars are capable of, but back then one can imagine a lot more skepticism.

Just as there is a ton of skepticism today of what LLMs can achieve. A competition like this clearly demonstrates where the tech is, and what is possible.

> there's no clear mapping to say how the LLM's compute should be limited to make a comparison

There is a very clear mapping of course. You give the same wall clock time to the computer you gave to the humans.

Because what it is showing is that the computer can do the same thing a human can under the same conditions. With your analogy here they are showing that there is such a thing as a car and it can travel 100 meters.

Once it is a foregone conclusion that an LLM can solve the ICPC problems and that question has been sufficiently driven home to everyone who cares we can ask further ones. Like “how much faster can it solve the problems compared to the best humans” or “how much energy it consumes while solving them”? It sounds like you went beyond the first question and already asking these follow up questions.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by furyofantares on Wednesday September 17, 2025 @09:00PM (#45283587)
You're right, they did limit to 5 hours and, I think, 3 models, which seems analogous at least.

Not enough to say they "won gold". Just say what actually happened! The tweets themselves do, but then we have this clickbait headline here on HN somehow that says they "won gold at ICPC".

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by Vegenoid on Thursday September 18, 2025 @12:54AM (#45285578)
Agreed. The linked messaging is much more clear: "achieved gold-medal level performance". This clearly separates them from competing against humans, which they didn't do, because their constraints are very different. The "AI wins gold at ICPC" line really does seem designed to rile people up.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by guyomes on Thursday September 18, 2025 @07:52AM (#45288507)
> under the same conditions

That's a very interesting question. When comparing wildly different computing machines, how to make a fair comparison?

At least two criteria comes in mind: the volume and the energy consumption.

Indeed we can safely assume that more volume and more energy leads to more computation power. For example, it is not fair to compare a 10m^3 room filled with computers with 10cm^3 computer. The same goes with the number of kilowhat-hours used.

Thinking further on those two criteria for GPUs and humans, we could also consider the access to energy and volume. First, energy access for machines has dramatically increased since the industrial revolution. Second, volume access for machines has also increased since the beginning of the mass production. In particular, creating one cube meter of new GPUs is faster than giving birth to a new human.

tldr: fair comparison of two machines should take into account their volume and their energy consumption. On the other hand, this might be mitigated by how fast a machine can increase its volume, and what is its bandwidth for energy consumption.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by hnfong on Wednesday September 17, 2025 @10:42PM (#45284569)
All the while with skeptics snarkily commenting "Cars can move fast, but they can't really run like a human!"
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by uludag on Thursday September 18, 2025 @01:41AM (#45285875)
... in opposition to the car makers who want to turn everything into highways and parking lots, who really want all forms of human walking to be replaced by automobiles.

"They really cant run like a human," they say, "a human can traverse a city in complete silence, needing minimal walking room. Left unchecked, the transitions to cars would ruin our city. So lets be prudent when it comes to adopting this technology."

"I'll have none of that. Cars move faster than humans so that means they're better. We should do everything in our power to transition to this obviously superior technology. I mean, a car beat a human at the 100m sprint so bipedal mobility is obviously obsolete," the car maker replied.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by apstls on Thursday September 18, 2025 @04:56AM (#45287270)
This metaphor drops some pretty key definitional context. If the common belief prior to this race was that cars could not beat horses, maybe someday but not today, then the article is completely reasonable, even warranted.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by Swizec on Wednesday September 17, 2025 @06:36PM (#45282254)
> Firstly, automobiles are really impressive. Second, with that out the way, these cars are not playing the same game as horses

Yes. That’s why cars don’t compete in equestrian events and horses don’t go to F1 races.

This non-controversial surely? You want different events for humans, humans + computers, and just computers.

Notice that self driving cars have separate race events from both horses and human-driven cars.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by throw310822 on Wednesday September 17, 2025 @07:23PM (#45282680)
I think you missed that the whole point of this race was:

"did we build a vehicle faster than a horse, yes/no?"

Which matters a lot when horses are the fastest land vehicle available. (We're so used to thinking of horses as a quaint and slow mean of transport that maybe we don't realize that for millennia they've been the fastest possible way to get from one place to another.)

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by Swizec on Wednesday September 17, 2025 @08:14PM (#45283177)
> "did we build a vehicle faster than a horse, yes/no?"

Yeah fair. There's also that famous human vs horse race that happens every few years. So far humans keep winning (because it's long distance)

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by dwohnitmok on Wednesday September 17, 2025 @10:11PM (#45284248)
If you're talking about the Man versus Horse Marathon (https://en.wikipedia.org/wiki/Man_versus_Horse_Marathon) it's the other way around. Overwhelmingly the horses win. Only occasionally does the human.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by Swizec on Wednesday September 17, 2025 @10:24PM (#45284384)
I stand corrected. My memory garbled that. Thanks!
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by gxs on Wednesday September 17, 2025 @07:19PM (#45282635)
Yeah I think the only thing OP was passing judgement on is on the competition aspect of it, not the actual achievement of any human or non human participant

That’s how I read it at least - exactly how you put it

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by in-silico on Wednesday September 17, 2025 @08:19PM (#45283215)
The point is that up until now, humans were the best at these competitions, just like horses were the best at racing up until cars came around.

The other commenter is pointing out how ridiculous it would be for someone to downplay the performance of cars because they did it differently from horses. It doesn't matter if they did it using different methods, that fact that the final outcome was better had world-changing ramifications.

The same applies here. Downplaying AI because it has different strengths or plays by different rules is foolish, because that doesn't matter in the real world. People will choose the option that that leads to the better/faster/cheaper outcome, and that option is quickly becoming AI instead of humans - just like cars quickly became the preferred option over horses. And that is crazy to think about.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by thrwaway55 on Wednesday September 17, 2025 @10:17PM (#45284295)
I feel the main difference is cars can't compress time in the way an array of computers can. I could win this competition with an infinitely parallel array of random characters typed by infinite monkeys on infinite typewriters instantly since one of them would be perfectly right given infinite submissions. When I make my tweet I would pick a single monkey cus I need infinite money to feed my infinite workforce and that's more impressive clearly.

Now obviously it's more impressive as they don't have infinite compute and had finite time but the car only has one entry in each race unless we start getting into some anime ass shit with divergent timelines and one of the cars (and some lesser amount of horses) finishing instantly.

To your last point we don't know that this was cheaper since they don't disclose the cost. I would blindly guess a mechanical turk for the same cost would outperform at least today.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by exclipy on Wednesday September 17, 2025 @11:43PM (#45285097)
In what way did the computer compress time? It completed it in 5 hours and I'm pretty sure they didn't invent a time machine
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by thrwaway55 on Thursday September 18, 2025 @03:54AM (#45286832)
How long does a single thread take to do an attempt? How long do two threads take? I don't want to assume people reading this forum are children.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by thornewolf on Thursday September 18, 2025 @04:00PM (#45294261)
This doesn't matter. I don't intend to be rude, of course. I believe this doesn't matter at all.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by in-silico on Friday September 19, 2025 @05:24AM (#45299653)
Considering that OpenAI's model got a higher score than any of the world's best collegiate programming teams, I'd guess that a mechanical turk would not do better (even if you gave them quite a bit of time).
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by lbrandy on Wednesday September 17, 2025 @06:30PM (#45282198)
I was struck how the argument is also isomorphic to how we talked about computers and chess. We're at the stage where we are arguing the computer isn't _really_ understanding chess, though. It's just doing huge amounts of dumb computation with huge amounts of opening book and end tables and no real understanding, strategy or sense of whats going on.

Even though all the criticism were, in a sense, valid, in the end none of it amounted to a serious challenge to getting good at the task at hand.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by LaffertyDev on Wednesday September 17, 2025 @06:31PM (#45282210)
I don’t think you’ll find many race tracks that permit horses and cars to compete together.

(I did enjoy the sarcasm, though!)

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by GoatInGrey on Wednesday September 17, 2025 @07:19PM (#45282637)
Snark aside, I would expect a car partaking in a horse race to beat all of the horses. Not because it's a better horse, but because it's something else altogether.

Ergo, it's impressive with nuance. As the other commenter said.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by bgwalter on Wednesday September 17, 2025 @06:42PM (#45282319)
The massive amounts of compute power is not the major issue. The major issue is unlimited amount of reference material.

If a human can look up similar previous problems just as the "AI" can, it is a huge advantage.

Syzygy tables in chess engines are a similar issue. They allow perfect play, and there is no reason why a computer gets them and a human does not (if you compare humans against chess engines). Humans have always worked with reference material for serious work.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by chpatrick on Wednesday September 17, 2025 @07:45PM (#45282888)
Humans are allowed to look up and learn from as many previous problems as they want before the competition. The AI is also trained on many previous problems before the competition. What's the difference?
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by bgwalter on Wednesday September 17, 2025 @07:53PM (#45282972)
I don't know what to say. Are you seriously positing that it is interesting that computers have better recall than humans?

Perhaps this helps:

https://timesofindia.indiatimes.com/technology/tech-news/exp...

https://mathstodon.xyz/@tao/114881418225852441

Especially:

"* The team leader gives the students unlimited access to calculators, computer algebra packages, formal proof assistants, textbooks, or the ability to search the internet."

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by chpatrick on Wednesday September 17, 2025 @08:07PM (#45283098)
It doesn't say anywhere that Gemini used any of those things at ICPC, or that it used more real-world time than the humans.

Also, who cares? It's a self contained non-human system that could solve an ICPC problem it hasn't seen before on its own, which hasn't been achieved before.

If there was a savant human contestant with photographic memory who could remember every previous ICPC problem verbatim and can think really fast you wouldn't say they're cheating, just that they're really smart. Same here.

If there was a man behind the curtain that was somehow making this not an AI achievement then you would have a point, but there isn't.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by raspasov on Wednesday September 17, 2025 @08:57PM (#45283552)
I think "hasn't seen before" is a bit of an overstatement. Sure, the problem is new in the literal sense that it does exist verbatim elsewhere, but arguably, any competition problem is hardly novel: they are all some permutation of problems that exist and have been solved before: pathfinding, optimization, etc. I don't think anyone is pretending to break new scientific ground in 5 hours.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by chpatrick on Thursday September 18, 2025 @06:31AM (#45287933)
It's not new scientific ground but a machine beating a challenging computer science problem unassisted is a big deal. If they can do that then there are a lot of other challenging things they can do.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by raspasov on Thursday September 18, 2025 @03:41PM (#45294055)
Like what exactly? As far as I can tell, the drug discovery is fizzling out, so it's not talked about much. Toxicity, for one, is a big problem, and the AI is not going to tell you whether the new drug it just concocted is suitable for humans or not.

Small model solves an easy problem; big model solves a challenging problem. I wouldn't call those problems; they are more like invented puzzles. Perfect match for the AI marketing department to "solve".

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by chpatrick on Friday September 19, 2025 @08:51PM (#45308666)
Whatever you say.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by raspasov on Wednesday September 24, 2025 @09:57PM (#45368456)
Have an AI model prove that P is not equal to NP. Then I’ll be impressed.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by chpatrick on Monday September 29, 2025 @11:01AM (#45414694)
So you'll only be impressed if it's superintelligent and solves a problem no human has? That's a ridiculously high bar.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by raspasov on Saturday October 04, 2025 @10:37PM (#45478465)
Humans have been shown to solve problems and discover things that "no human has" up to that point. So I wouldn't even call it "superintelligent". But it would definitely be truly useful at that point!

The marketing speak of some companies goes into effectively fantastical territory. Some claims that are made effectively imply P = NP, which is like bad TV-series-level science fiction, but some people fall for it.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by melenaboija on Wednesday September 17, 2025 @08:44PM (#45283424)
Comparing power with reasoning does not make any sense at all.

Humans have surpassed their own strength since the invention of the lever thousands of years ago. Since then, it has been a matter of finding power sources millions of times greater such as nuclear energy

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by Gud on Thursday September 18, 2025 @02:17AM (#45286125)
Your analogy is flawed.

Are the humans allowed to bring their laptops and use the internet? Or a downloaded copy?

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by LunaSea on Thursday September 18, 2025 @05:07AM (#45287345)
Power is one thing, efficiency is another.

Humans are more efficient watt for watt than any AI ever invented.

Now if you were to limit AIs to 400 watts we could probably thinks it's fair.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by matheusd on Thursday September 18, 2025 @06:02AM (#45287743)
> Humans are more efficient watt for watt than any AI ever invented.

Indeed they are. For now. The long term trend is not in our favor.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by LunaSea on Thursday September 18, 2025 @07:35AM (#45288368)
I disagree, the long term trend is that we do not have the available electricity and water for this to continue for much longer.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by pedrosorio on Friday September 19, 2025 @12:22AM (#45297921)
Regarding electricity, it depends on what you mean by “we”, I guess

https://www.voronoiapp.com/energy/-China-Generated-More-Elec...

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by rich_sasha on Thursday September 18, 2025 @02:14PM (#45293033)
There's a difference. How much money went into training the computer here Vs the human? If you want to prove that a computer can, at extreme cost and effort, beat a human - sure, it's possible.

But you can also conclude that putting a lot of money and effort pays off. It's more like comparing a horse to a Ferrari that had millions of development costs, has a team of engineers maintaining it, isn't reusable, and just about beats Chestnut. It's a long way until the utility of both is matched.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by j_timberlake on Thursday September 18, 2025 @05:35PM (#45295296)
This response is good but the more general problem is that people are in "It doesn't look like anything to me" mode like Westworld robots seeing advanced technology. If there's a way to snap people out of that, I've never seen it.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by paladin314159 on Wednesday September 17, 2025 @05:01PM (#45281350)
> I think that, if you gave me the ability to search the pre-contest Internet and a week to prepare my submissions, I would be kind of embarrassed if I didn't get gold, and I'd find the contest to be rather less interesting than I would find the real thing.

I don't know what your personal experience with competitive programming is, so your statement may be true for yourself, but I can confidently state that this is not true for the VAST majority of programmers and software engineers.

Much like trying to do IMO problems without tons of training/practice, the mid-to-hard problems in the ICPC are completely unapproachable to the average computer science student (who already has a better chance than the average software engineer) in the course of a week.

In the same way that LLMs have memorized tons of stuff, the top competitors capable of achieving a gold medal at the ICPC know algorithms, data structures, and how to pattern match them to problems to an extreme degree.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by amluto on Wednesday September 17, 2025 @05:20PM (#45281529)
> I can confidently state that this is not true for the VAST majority of programmers and software engineers.

That may well be true. I think it's even more true in cases where the user is not a programmer by profession. I once watched someone present their graduate-level research in a different field and explain how they had solved a real-world problem in their field by writing a complicated computer program full of complicated heuristics to get it to run fast enough and thinking "hmm, I'm pretty sure that a standard algorithm from computer graphics could be adapted to directly solve your problem in O(n log n) time".

If users can get usable algorithms that approximately match the state of the art out of a chatbot (or a fancy "agent") without needing to know the magic words, then that would be amazing, regardless of whether those chatbots/agents ever become creative enough to actually advance the state of the art.

(I sometimes dream of an AI producing a piece of actual code that comes even close to state of the art for solving mixed-integer optimization problems. That's a whole field of wonderful computer science / math that is mostly usable via a couple of extraordinarily expensive closed-source offerings.)

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by avcxz on Wednesday September 17, 2025 @05:34PM (#45281661)
> That's a whole field of wonderful computer science / math that is mostly usable via a couple of extraordinarily expensive closed-source offerings.

Take a look at Google OR-Tools: https://developers.google.com/optimization/

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by amluto on Wednesday September 17, 2025 @05:53PM (#45281839)
OR-Tools is a whole grab-bag of tools, most of which are wrappers around various solvers, including Gurobi and CPLEX. It seems like CP-SAT is under the OR-Tools umbrella, and CP-SAT may well be state-of-the-art for the specific sets of problems that it's well-suited for.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by bubblyworld on Thursday September 18, 2025 @01:59AM (#45285982)
Yeah, absolutely, I spent months preparing for the ICPC with my team and ended up scoring a paltry 3/12. A week would likely not have helped us at all - we simply had no idea how to approach the rest! Top teams are on another level.

Compute and such is a fair point but that AI is here at all is mind-blowing to me.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by roadside_picnic on Wednesday September 17, 2025 @08:05PM (#45283084)
> whereas the teams are allowed to bring a 25-page PDF

This is where I see the biggest issue. LLMs are first-and-foremost text compression algorithms. They have a compressed version of a very good chunk of human writing.

After being text compression engines, LLMs are really good at interpolating text based on the generalization induced by the lossy compression.

What this result really tells us is that, given a reasonably well compressed corpus of human knowledge, the ICPC can be view as an interpolation task.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by Rudybega on Wednesday September 17, 2025 @09:02PM (#45283610)
If we develop a system that can:

- compress (in a relatively recoverable way) the entire domain of human knowledge

- interpolate across the entire domain of human knowledge

- draw connections or conclusions that haven't previously been stated explicitly

- verify or disprove those conclusions or connections

- update its internal model based on that (further expanding the domain it can interpolate within)

Then I think we're cooking with gasoline. I guess the question becomes whether those new conclusions or connections result in a convergent or divergent increase in the number of new conclusions and connections the model can draw (e.g. do we understand better the domains we already know or does updating the model with these new conclusions/connections allow us to expand the scope of knowledge we understand to new domains).

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by andoando on Wednesday September 17, 2025 @11:16PM (#45284843)
You can use the same framing for human reasoning except its over visual/auditority/spatial data and not just text.

You don't remember every detail of what you've seen correct? You store some lossy compression like "I went to a park"

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by OtherShrezzing on Wednesday September 17, 2025 @07:53PM (#45282975)
The human teams also get limited to one computer shared between 3 people. The models have access to an effectively unbounded number of computers.

My argument does feel a bit like the “Watson doesn’t need to physically push the button” equivalents from when that system beat Jeopardy for the first time. I assume 5 hours on a single high-end Mac would probably still be enough compute in the near future.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by amluto on Wednesday September 17, 2025 @10:46PM (#45284602)
I found the Watson match to be rather absurd. It would have been much more interesting if the rules had been modified so that all contestants had, say, two seconds two press the buzzer and that the contestant who got to answer first would be chosen by random selection among those who pressed the button. This would at least have made the competition be about who could come up with the most correct answers (questions).
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by Darmani on Thursday September 18, 2025 @06:32AM (#45287939)
The people saying that were wrong, BTW -- Watson did have to physically press a button.

https://www.wired.com/2011/02/ibm-watson-speed/

> __Brown: __ Watson has a mechanical button-presser. It uses the same signaling device [the button] that the human competitors use in the game. Once Watson has decided that it wants to ring in because it has found an answer with a high-enough confidence, and it receives the signal that the buzzers are open and you can ring in, it then has to trigger the mechanical button presser and mechanically press the button.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by OtherShrezzing on Thursday September 18, 2025 @07:18AM (#45288238)
Watson originally just sent an electronic signal. The physical button pushing was introduced later, to stave off the criticism that it had an unfair advantage.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by modeless on Wednesday September 17, 2025 @05:06PM (#45281396)
It doesn't matter how many instances were running. All that matters is the wall clock time and the cost.

The fact that they don't disclose the cost is a clue that it's probably outrageous today. But costs are coming down fast. And hiring a team of these guys isn't exactly cheap either.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by zeroonetwothree on Wednesday September 17, 2025 @05:09PM (#45281435)
Human teams are limited to three people. So why doesn’t it matter how many instances they used?
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by modeless on Wednesday September 17, 2025 @05:11PM (#45281448)
Human brains and cloud instances are not remotely equivalent. What you can compare on an equivalent basis is cost.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by LunaSea on Thursday September 18, 2025 @05:08AM (#45287358)
Or energy usage.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by modeless on Thursday September 18, 2025 @05:38PM (#45295321)
Cost is inclusive of energy and a lot more. It's a better metric.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by ben_w on Wednesday September 17, 2025 @05:26PM (#45281579)
All instances of any given model are kinda the same, for lack of a better word, "person": same knowledge, same skills, same failings.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by warkdarrior on Wednesday September 17, 2025 @06:47PM (#45282365)
I bet with human teams it'll take longer to solve a problem the more people you have on the team.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by kenjackson on Wednesday September 17, 2025 @09:13PM (#45283687)
This is what the argument is? 10 years ago if you said you could do this with every computer on the planet and every computer scientist focused on trying to create the code to do this I would’ve given you absurd odds against it getting 12 problems right on ICPC. 10 years ago it couldn’t even reliably parse the question statement.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by theragra on Wednesday September 17, 2025 @06:04PM (#45281939)
I think your analogy is lacking. Human brain is much more efficient, so it is not right to say "giving a human team a week instead of five hours". Most likely, the whole OpenAI compute cannot match one brain in terms of connections and relations and computation power.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by stevenhuang on Wednesday September 17, 2025 @06:51PM (#45282392)
As always with these comparisons you neglect to account for the eons necessary for evolution to create human brains.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by GoatInGrey on Wednesday September 17, 2025 @07:23PM (#45282682)
But as a product of evolved organisms, LLMs are also a product of evolution. They also came several hundreds of thousands of years later.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by vrighter on Thursday September 18, 2025 @08:44AM (#45288943)
so in that case, don't compare them to humans in the first place
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by _diyar on Wednesday September 17, 2025 @05:37PM (#45281687)
I think your assessment is spot on. But I also think there's a bigger picture that's getting lost in the sauce, not just in your comment but in the general discourse around AI progress:

- We're currently unlocking capabilities to solve many tasks which could previously only be solved by the top-1% of the experts in the field.

- Almost all of that progress is coming from large scale deep learning. Turns out transformers with autoregression + RL are mighty generalists (tho yet far from AGI).

Once it becomes cheap enough so the average joe can tinker with models of this scale, every engineering field can apply it to their niche interest. And ultimately nobody cares if you're playing by the same rules as humans outside of these competitions, they only care that you make them wealthy, healthy and comfy.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by dist-epoch on Wednesday September 17, 2025 @05:21PM (#45281542)
If you want to play that game, let's compute how much energy was spent to grow, house and educate one team since they were born, over 20 years against how much was spent training the model.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by somewholeother on Wednesday September 17, 2025 @06:47PM (#45282366)
This is a fair analogy, but let's also consider that these human beings weren't designed with the express purpose of becoming experts in their field and performing in this way for this specific purpose (albeit in a generalist manner).

We are most definitely in agreement about the folly of comparing the abilities of LLMs to humans, since LLMs are to a greater extent the product of much collective human endeavour. "Living memories" would perhaps be a better description of their current state, and their resultant impact on the human psyche.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by vrighter on Thursday September 18, 2025 @08:49AM (#45288995)
a human being uses about 1 gWh in a lifetime on average.

Training one model nowadays uses more than that. So the model also cost more to train than to raise a team member (possibly all three of them)

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by vrighter on Thursday September 18, 2025 @09:28AM (#45289450)
whoops, got the numbers wrong. 1gWh is how much electricity a person uses, not how much energy the body consumes.

That works out to about 72 years365 days2000 kcal = approx 61 gWh

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by th0ma5 on Wednesday September 17, 2025 @07:30PM (#45282756)
Yes yes given this why didn't it do better and isn't it embarrassing to have done it through statistical brute force and not intelligence.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by tdb7893 on Wednesday September 17, 2025 @09:23PM (#45283774)
As someone who has been to the ICPC finals around a decade ago I agree that the limited time is really the big problem that these machine learning models don't really experience in the same way. Though that being said these problems are hard, the actual coding of the algorithms is pretty easy (most of the questions use one of a handful of algorithms that you've implemented a hundred times by the time you're in the finals) but recognizing which one will actually solve the problem correctly is not obvious at all. I know a lot of people that struggled in their undergrad algorithms class and I think a lot of those people given the ICPC finals problems would struggle even with being able to research.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by dddgghhbbfblk on Wednesday September 17, 2025 @09:47PM (#45283977)
I think that's because the framing around this (and similar stories about eg IMO performances) is imo slightly wrong. It's not interesting that they can get a gold medal in the sense of trying to rank them against human competitors. As you say, the direct comparisons are, while not entirely meaningless, at least very hard to interpret in the best of cases. It's very much an apples to oranges situation.

Rather, the impressive thing is simply that an AI is capable of solving these problems at all. These are novel (ie not in training set) problems that are really hard and beyond the ability of most professional programmers. The "gold medal" part is informative more in the sense that it gives an indication of how many problems the AI was able to solve & how well it was able to do them.

When talking with some friends about chatgpt just a couple years ago I remember being very confident that there was no way this technology would be able to solve this kind of novel, very challenging reasoning problem, and that there was no way it would be able to solve IMO problems. It's remarkable how quickly I've been proven wrong.

Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by ragequittah on Wednesday September 17, 2025 @11:30PM (#45284977)
It feels like half of the people I see talk about AI are still under the impression it's a spicy autocomplete. If you use a SOTA model for a week and still feel this way your bias must be very strong.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by dddgghhbbfblk on Thursday September 18, 2025 @12:46AM (#45285517)
I must be missing something because I don't understand how this is related to my comment.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by m3kw9 on Wednesday September 17, 2025 @09:59PM (#45284104)
the end game is that running similar tasks at any moment time and place.
Re:DeepMind and OpenAI Win Gold at ICPC, OpenAI AKs by theywillnvrknw on Thursday September 18, 2025 @07:09AM (#45288181)
The more important question I'd like answered is did the models have the same number of attempts (submissions) as human contestants.

If not were penalities applied for multiple solutions.