edit: >> https://techcrunch.com/2025/10/19/openais-embarrassing-math/
The ability to find incredibly obscure facts and recall them to solve "officially unsolved" problems in minutes is like Google Search on steroids. In some sense, it is one core component of "deep expertise", and humans rely on the same methodology regularly to solve "hard" problems. Many mathematicians have said that they all just use a "bag of tricks" they've picked up and apply them to problems to see if they work. The LLMs have a huge bag of very obscure tricks, and are starting to reach the point that they can effectively apply them also.
I suspect the threshold of AGI will be crossed when the AIs can invent novel "tricks" on their own, and memorise their own new approach for future use without explicitly having to have their weights updated with "offline" training runs.
I agree.
Although I don't think the current AI training method will be able to do AGI etc. (I agree with Yann LeCun's POV)
Being young, I don't know how I feel about the philosophical idea of AGI.
Are you asking me how LLMs work?
The theory proposed by the original commenter was that there could have been some secret training data the model was trained on that made it possible to solve this problem set. So the only conclusion is they are implying it's a conspiracy by OpenAI to hide some novel math research they funded merely to do marketing about solving math problems (then convincing multiple math experts to verify and support it with papers). That is the definition of a conspiracy.
In all seriousness though: My suggestion is that those shepherding the frontier of AI start acting with more transparency, and stop acting in ways that encourage conspiratorial thinking. Especially if the technology is as powerful as they market it as.
That is not true and a complete misrepresentation of recent progress of AI in math. It is therefore not necessary to believe the conspiracy theory you described in order to explain recent progress of AI in math.
Congrats to the OpenAI team for one of the most significant breakthrough discoveries in AI history.
I'll gladly admit I think what these companies are doing is unethical, and I'm sure that biases my thinking toward skepticism.
That said, there remains way too much that is hidden to be able to effectively evaluate what is going on. You have the perfect storm:
- AI companies do not share their custom internal harnesses.
- AI companies do not share their custom internal training data.
- AI companies do not share how much compute they allocate to trying to solve problems of this nature.
- AI companies are primarily marketing their models to investors as human-replacing rather than human-augmenting.
- AI companies are under enormous financial pressure to make their business work.
Is it likely that they literally had a mathematician discover this, put it into the training data, and then prompted it out? Of course not.
But it would make a world of difference--in evaluating the impressiveness of this discovery and LLM capabilities in general--if we were to know the extent to which the training data crosses over this problem, the harness with which this was ran, and how much compute was spent.
Until they bring more transparency to the whole process--something which some of the mathematicians commenting on this even asked for--I will personally take discoveries of this nature with a good dose of salt.
Really? Any references to read more?
- https://www.theverge.com/cs/features/831818/ai-mercor-handshake-scale-surge-staffing-companies
- https://outlier.ai/math/en-us
- https://www.opentrain.ai/
- https://www.pin.com/blog/ai-labs-hiring-train-models/
Does this expert data likely contain this proof within it? No. Would it temper the impressiveness to know they have a large amount of novel mathematical training data, an internal Lean harness for evaluation of open conjectures, and spent hundreds of millions in compute to calculate this? Yes.
Also why pay anyone, when they can keep up with all the papers that not one man can read them all? That seems to me like wasted money.
Another point is that that's not how AI training works anyway. It's much easier to put it in context rather than re-train them with every bit of random maths you find out. Things at the tail-end of the power law doesn't stick. At least, last I checked...
They pay people for expert training data they do not share because it gives them an edge over other AI companies. And, as always, deep learning is enormously data-hungry, and we've gotten to the point where publicly available data has been exhausted.
AI companies absolutely retrain models regularly to keep up with the cutting edge. There is a reason why this announcement references an internal, unreleased model, rather than "we just put a lot of new math papers within the GPT5.5 context window and found this."
Without knowing all this model has been trained on though, it is pretty hard to ascertain the extent to which it arrived to this "on its own". The entire AI industry has been (not so secretly) paying a lot of experts in many fields to generate large amounts of novel training data. Novel training data that isn't found anywhere else--they hoard it--and which could actually contain original ideas.
It isn't likely that someone solved this and then just put it in the training data, although I honestly wouldn't put that past OpenAI. More interesting though is the extent to which they've generated training data that may have touched on most or all of the "original" tenets found in this proof.
We can't know, of course. But until these things are built in a non-clandestine manner, this question will always remain.