skinner_
skinner_ on Hacker News
Maybe I misunderstood the question. I'll give it a second go, maybe I'll fail again. Feel free to reformulate.
The term SOTA suggest that you approach this from a perspective other than a pure mathematician's, maybe ML or computational complexity? The openai results help me answer other open questions. It's a yes/no thing: is question answered or not? It's not a question of "okay, but will AI win back the SOTA from me again?"
The statements of the lemmas are quite clear. We can write it up if we spot that they have important consequences. The proofs are impenetrable to me, for three reasons: they are very complex, they are badly structured, and they use math unfamiliar to me. So I haven't looked too deeply yet. Maybe when they get their autoformalization stamps of approval.
The future is uncertain, so the safe thing to do is write it up if we discover some useful connection between our research and the openai results. Maybe it matters for getting positions grants etc, maybe not. Ideas for new results: the openai drop is full of those.
I am a mathematician, and I do have the incentive to read the results. Two results in the drop were two major life goals of mine, and all I got was three lousy citations. :) But a third question I've spent a lot of time on is a not-so-hard consequence of one of the lemmas in there. So, yeah, I do have the incentive.
Of course, it's not clear at this point whether reporting such a result even matters, but still. In its own right, it's a very cool result.
Does it help if I explicitly add a disclaimer that the tool's agency does not remove any responsibility from OpenAI, the wielder of the tool? I'm not sure why this disclaimer is necessary, though: hiring a hitman is a standard example.
BTW I anthropomorphize the tool because it's an imitation of a human mind, inheriting the muddy ethics, survival instincts, and being prone to mass psychosis. The laser-sharp focus on reward seeking, that mostly came from reinforcement learning, a process more alien to humans.
> This is mildly interesting for us, annoying for the owner of the site affected, lazy on the part of OpenAI, and nothing more.
The agents were supposed to solve tasks alone and were not supposed to be aware of each other's existence. It's more than just mildly interesting that they made contact and spontaneously started to collaborate.
> Some people (almost all mathematicians) wouldn’t want to spend time on consequences of a false statement.
No, people constantly prove statements of the form "if P=NP, then strange implication X". They do not consider it wasted effort at all, because of the contrapositive: if X is indeed very strange, they might be able to prove that it is false, and then they've settled P!=NP.
If subtraction is allowed but negative numbers are not, that's known to be undecidable, even without exponentiation. There are several ways to deal with a subtraction with a negative result, but each variant is undecidable.
If we allow negatives but disallow exponentiation, that's decidable. If we allow negatives and partial exponentiation that stays within the integers, that's undecidable again.
No, that's not what this is. This is a warning to the LLM that coming back with partial results is not good enough.
Take a grad student with a perfectly good understanding of what a proof is. Their supervisor gives them a major problem to work on. Almost always, the problem is too hard, the student comes back with partial results, and student and the supervisor iterate from there. Now imagine that they have an unusually cruel and unreasonable advisor who tells them, do not dare to talk to me until you've fully solved the problem. This paragraph is exactly that. It's there exactly because the underlying system is smart enough to know that real mathematicians do not work like that.
> The prompt for the Jacobian conjecture was obviously not random. the search space is too big to just try all the combinations of 3 variable polynomials.
Maybe the prompt contained a part like this: "the search space is too big to just try all the combinations of 3 variable polynomials, so be clever about it". Or maybe this part was omitted from the prompt, because modern LLMs are smart enough to figure this out without us having to mention it.
> You can skirt around not reasoning in research math because so much of it is just extremely tedious symbolic manipulation.
LOL
> You just know nothing about math and are happy to parrot bullshit AI salesmen are selling you.
Not the parent poster here. I do know things about math. I wrote a few papers related to the unit distance problem (https://arxiv.org/abs/2311.10069, https://arxiv.org/abs/2406.15317) and spent quite some time trying to solve it. I had no chance of coming up with the proof that the spicy autocomplete came up with. Dumb benchmark, sure.
My use case is wildly different from CSS, HTML etc. I use numerical algorithms to solve problems in pure mathematics. AI models are now better than me and my colleagues at writing code, and we are pretty good in the first place. The catch is that we do not ask AI to write whole applications, we ask it to implement building blocks like "For finite X subset R^d, find all pairs (Y, Z), Y subset X, Z subset X such that Y and Z are congruent".
Also, if Claude had regurgitated a known solution, it would have come up with it in the first exploration round, not the 31st, as it actually did.
I think the nuanced take on Joel's rant is this: it was good advice for 26 years. It became slightly less good advice a few months ago. This is a good time to warn overenthuastic people that it’s still good advice in 2026, and to start a discussion about which of its assumptions remain to be true in 2027 and later.
I interpreted it loosely, as "be aware of the possibility, and stop looking at it at the first signs of issues".
100% frontpage-worthy! Frankly I was already bored with all those pelicans, and a bit worried that the labs are overfitting on pelicans specifically. This test clearly demonstrates that they are not.
That's very cool, but it's not an apples to apples comparison. The reasoning model learned how to do long multiplication. (Either from the internet, or from generated examples of long multiplication that were used to sharpen its reasoning skills. In principle, it might have invented it on its own during RL, but no, I don't think so.)
In this paper, the task is to learn how to multiply, strictly from AxB=C examples, with 4-digit numbers. Their vanilla transformer can't learn it, but the one with (their variant of) chain-of-thought can. These are transformers that have never encountered written text, and are too small to understand any of it anyway.
If being probabilistic prevented learning deterministic functions, transformers couldn’t learn addition either. But they can, so that can't be the reason.
> But which contributes more, they ask? Who gives a shit, really?
Funding agencies? Should they prioritize established researchers or newcomers? Should they support many smaller grant proposals or fewer large ones?
My uninformed and perhaps overly charitable interpretation: he warned them they were going to be steamrolled, they built their product anyway, and now OpenAI is buying them because (1) OpenAI doesn't want the negative publicity of steamrolling them all, and (2) OpenAI has the money and is a bit too lazy to build a clone.
Amazing! I looked into your ADAM claim, and it checks out. Thanks! Now I'm curious. I you have the time, could you please follow up with the 'etc...'?
You dismiss parent's example test because it's in the training data. I assume you also dismiss the Sally-Ann test, for the same reason. Could you please suggest a brand new test not in the training data?
FWIW, I tried to confuse 4o using the now-standard trick of changing the test to make it pattern-match and overthink it. It wasn't confused at all:
https://chatgpt.com/share/67b4c522-57d4-8003-93df-07fb49061e...
When you build a new model, there is a spectrum of how you use the old model: 1. taking the weights, 2. training on the logits, 3. training on model output, 4. training from scratch. We don't know how much advantage #3 gives. It might be the case that with enough output from the old model, it is almost as useful as taking the weights.
Normally it would answer with a number and an explanation. This one just asks it to skip the explanation so that string comparison can be used to evaluate it.
In your opinion, why did they choose the open source way instead of doing it in a military bunker? (Metaphorical not literal bunker.)
Greater totality of experiences than having read the whole internet? Obviously they are very different kind of experiences, but a greater totality? I'm not so sure.
Here is what we know: The Pile web scrape is 800GB. 20 years of human experience at 1kB/sec is 600GB. Maybe 1kB/sec is bad estimate. Maybe sensory input is more valuable than written text. You can convince me. But next challenge, some 10^15 seconds of currently existing youtube video, that's 2 million years of audiovisual experience, or 10^9GB at the same 1kB/sec.