qnleigh

That was a really nice talk. Most of it would be understandable to an undergraduate math major. Some interesting quotes:

> [shows a slide with 3 sentences on it] If somebody tweeted this to me from the future, I would know what to do with this... Every step is very natural, and you just know how to continue.

> Everything was basically known, it's just that nobody tried this combination. Everybody believed that this is a combination that has a very realistic combination of working soon... [lists a sequence of technical but standard steps that were used in the proof] Exploiting the Gauss sum that resonates with Mobius, that's a genuinely clever step, but which Iwaniec [sp] observed, but which he actually intended to exploit for this purpose. So this was clever but was actually known by some people... And then there's the final hope that you can take all of this and obtain quasi-RH, and I think this is probably where more people were checking out, because it seemed a bit like a stretch.

> It turned out that [quasi-RH] came from tools we already had, but we somehow lost the confidence. So I think this is one of the big advantages of AI - it is not subject to psychology. It tries a lot of hopeless things, and it tries them with enthusiasm.

> This method caps out at 3/4 [meaning that it cannot suffice to prove the full Riemann hypothesis].

Look at the reasoning the models give before you get too excited here. For the Pong-vs-Pacman example, the model uses observations like small weights for certain arrow keys and large weights where user-controlled objects are. For the language example, it again recognizes some very shallow patterns (Romaji has a limited number of letter groupings) and is taking a 3-option multiple choice test. It's a neat result, but I think the introduction overstates the significance.

> I think the model helps mathematicians more than it hurts them

I would agree with you if these models were available to mathematicians, and OpenAI wasn't effectively competing with them by spending millions of dollars in compute to rush out solutions.

This moat issue comes up a lot, but look at what's been happening so far; open models are consistently 6-12 months behind. What if their moat is simply staying ahead?

And once their models start to accelerate their progress, which may already be happening, this would further solidify their lead.

The introduction is very nicely written, you're right. I learned a lot from it. There's some detail on how impractical their algorithm is:

> Our result is purely asymptotic: for arbitrary 3-CNF formulas with m = 5n clauses, the bound we prove does not improve on the 2n of exhaustive search unless n > 10^10000,

So their algorithm beats standard algorithms only when you have more variables than there are atoms in the universe (10^80). Even if every atom contained a whole universe, and each atom in each universe itself contained a whole universe, iterated 100 times, your still not there.

I thought it was interesting that in the original drop of papers, they announced much more progress on most other Millenium problems than on Yang-Mills existence. At the time I thought this might mean that it was a more stubborn problem, but it turns out that they had made a lot progress after all.

The Yang-Mills Millenium Problem has two parts though. The second part is to show that Yang-Mills theories have a mass gap, meaning that the lightest particle has non-zero mass. This is important because experimentally we know that there is a mass gap, so the theory must have one too to in order to actually be the correct model of nature. I wonder if they again have unannounced progress here.

It's hypothesized that a similar story played out for lignin, leading to the majority of the world's coal deposits. In the Carboniferous period ~300 million years ago, plants evolved a class of organic polymers called lignins which were difficult to break down. This lead to a massive accumulation of dead plant matter that nothing could digest. It only ended when fungi evolved the ability to break down lignin.

See the section on coal formation here https://en.wikipedia.org/wiki/Carboniferous

There's already some precedence for this in machine learning; DeepMind spun out Isomorphic Labs after the success of AlphaFold in solving the protein folding problem, hoping to use it for drug discovery.

LLMs haven't yet made a contribution to science or technology at the level of AlphaFold, but it looks like at least Anthropic is gunning for something like this with their AI-driven wetlab.

Probably they have either tried using LLMs to crack most cryptographic systems, or they have strict rules against doing so. Either way, the US govt. will probably make sure it gets a chance to do so before this internal model is released. There's a good chance they have started already.

Given the internal models that they have access to, I'm more worried about software companies being out of touch with the next generation of models will enable. Anecdotally I've seen that it takes a while for an organization to adjust to new LLM capabilities, even if everyone had access.

This article has basically nothing to do with the OpenAI paper; it's a critique of the entire field of computer science, rehashed to pick at one of their papers.

Wake me up when someone has understood the proof.

> This brings us to a big underlying problem with the way AI is being sold to us: it's being exaggerated to sound like way more than it is, and it's being sold to people/governments as something that will be capable of exceeding human "intelligence" in the near future when it's not capable of intelligence at all.

How can you possibly hold this belief after Tuesday? An LLM solved a significant number of major open problems in many subfields of mathematics, including a major breakthrough on the Riemann Hypothesis. Get your head out of the sand.

Terence Tao:

> And I mourn the path not taken, and the opportunities lost in the frantic race to develop this technology. Labs submitting their frontier models to independent researchers for proper scientific evaluation. Coordination with the research community to ensure these tools are applied to complement and enhance the abilities and activities of human researchers, rather than compete with them. Use of these tools to foster collaboration and sharing, rather than competition and secrecy. Opening new doors, without closing old ones.

But that is not the path we now find ourselves in. Instead, the community needs to come together more than ever. To clearly declare our own standards and values, to build our own tools and practices, to support our most vulnerable members, and to chart our own path forward. Let’s get to work.

Also this, from Peter Scholze:

We should remember that we are all in this together; mathematics is a marathon, not a sprint; and the goal is and always will be the human understanding of mathematics, which will invariably take time.

What is worrying me the most, at the moment, is actually the (in)security of cryptography. Finding algorithms breaking standard cryptographic protocols is a number theory problem whose difficulty does, to my non-expert eyes, not significantly exceed what these systems are now capable of. And it would have disastrous consequences on society if such an algorithm is found.

This is pretty insightful:

> One thing to keep in mind about this drop of papers is that they were not created by OpenAI with the main goal of making progress in mathematical research, or even primarily for PR (there were some PR posts, but e.g. the final announcement was comparatively restrained). The main reason these papers exist is that OpenAI used unsolved math problems to evaluate their internal models for the purpose of improving their capabilities. They had to use hard open problems because these are the only ones still posing a challenge to frontier AI models. I believe that, from OpenAI’s point of view, the papers are essentially a byproduct of their internal development.

I choose option 1. It's in the name! OpenAI should give mathematicians the chance to do at least some of this work themselves. It would be a lot easier for the community to adjust if they could use these same tools to explore how math research can adapt to the new circumstances. Instead, OpenAI is racing ahead to get as many results as possible with a proprietary model that the mathematical community cannot access. If these companies are building AI to benefit humanity as they assure us, then shouldn't they share their resources rather than vacuum up everything for themselves? Especially for something with as little direct monetary value as pure math.

I think there is way too much focus on how they released the results; it was going to be a huge shock no matter what. We should be talking about the massive concentration of "mathematical wealth" occurring here, and how it is a microscosm for what may happen on a much larger scale as AI outperforms humans on more and more domains.

> That doesn't take away from the fact that Math just turned 180° on its head and there's no going back.

100% agree. I'm not saying that it does.

These were all related to the Hodge conjecture, one of the Millennium Prize problems. There are still several other manuscripts related to the Hodge conjecture.

We'll see if there are any other gaps found in the coming months.

They just retracted 3 manuscripts:

In “Algebraicity of Weil classes on split abelian eightfolds” a sign error invalidates a stabilization-trace cancellation argument and the construction used by two dependent papers. As a result, we have withdrawn the following three manuscripts:

- Algebraicity of Weil classes on split abelian eightfolds

- Algebraicity of Kuga–Satake Correspondences for K3 Surfaces

- The rational Hodge conjecture for products of K3 surfaces

These were all related to the Hodge conjecture, one of the Millennium Prize problems. I wonder how many more will be retracted in the coming months.

https://github.com/openai/math/pull/1/files/301488868beec11b...

They have lots of mathematicians. Also there is a programming language called Lean that can check the correctness of a proof. The LLMs have written Lean proofs for many of their results, so we can be pretty sure that those are correct. For the rest it will take some time for people to check them. I wouldn't be surprised if a few turned out to have subtle gaps, but also these models are so capable that they might not.

This is a really interesting observation, especially given their current valuation and circumstances. Probably the verifiability of mathematics is enabling this progress, and making progress on science and engineering is not so straightforward. You can be sure that this is a top priority for them, especially now that there is concrete evidence that LLMs can achieve super-human performance at math. A model that achieves similarly super-human performance at things like material or drug design would be incredibly valuable. In fact, maybe this is why Anthropic hasn't been making such enormous strides in mathematics - they have been prioritizing biology instead.

If you scroll around on this thread, you'll see mathematicians discussing how they worked on one of these probably for a quarter century. Others arguing whether another is the biggest number theoretic breakthrough in 100 years or 200 years. It looks like they have made substantial progress on 4 Millennium Problems now. Unless it turns out that almost all of these results are wrong, what would it even mean for LLMs to be a dead end now? This is a historical day for mathematics and it will not be forgotten.

Interesting, that does look relevant. I don't have any sense how significant this is though (do you?)

It's notable that LLMs have now made substantial progress on four of the seven Millennium Prize problems, resolving one of them: Hodge, Birch-Swinnerton-Dyer, Riemann, and Navier-Stokes, which they resolved. No sign of P vs. NP or Yang-Mills existence and mass gap as far as I can tell, which is interesting.

Talking to some friends in physics this evening, most of the physics-related results that we could recognize were very mathematical, proving things rigorously where the physics community already had strong expectation. For instance, for a certain model of magnetism (the spin-1 Heisenberg chain), it was strongly expected that there is a finite energy gap between the ground state and the first excited state, but proving this rigorously was quite challenging. So while these are major results in mathematical physics, they probably don't rise to the level of a Millennium problem for the field.

It's interesting to think what a comparable breakthrough in physics might look like, since physics tends to favor things like conceptual understanding and applications over mathematical rigor. Maybe a new quantum algorithm, understanding of high-temperature superconductivity, a precise description of M theory...

Can you elaborate/give context? I haven't heard of this problem before, but curious to hear from someone who has.

I was talking with a number of physicists this evening, and of the papers who's problem statement we understood, I don't think any of them will have near-term applications. For most of them, they were results that I think the physics community believed to be true, and the paper provides the first rigorous proof. For several it was news to me that they weren't already theorems! These results are important advances in mathematical physics, but they don't tend to have much impact on experiment.

From what I can tell, all of the physics results here are quite mathematical. But I am very curious how the internal model they used would perform on more applied problems.

> it would be the single greatest advance in math

Did you mean to not qualify that? That is a bold statement indeed.

Can you name something specific that you think LLMs will not be able to achieve? Do you just mean consciousness or are there any specific intellectual tasks?

> After scratching the itch of knowing that something's been written completely rigorously, it's easy to call a work done when maybe we shouldn't call it done yet. And there's contexts where this is maybe OK, research comes to mind...

It's funny, because all of the advice he gives in the rest of the video would be amazing to apply to research papers as well. I've had people discourage me from writing research papers pedagogically because it is "unusual," which really meant that it would be looked down upon. It's kind of tragic that the norm is to write research papers that strip away the human details of how a definition is motivated or how something was discovered. This kind of information is usually only shared through in-person discussions or occasionally research presentations, but it is often the core knowledge that enables one to do research in a given field.

It's a great video though. He gives a little checklist of ways to check if your work is clear:

- Do definitions have motivating examples?

- Do proofs [or other results] feel rediscoverable?

- Is there personality?

- Are core ideas illustrated with diagrams?

- Is relative importance highlighted?

...

He also gives a beautiful example of how powerful it is to give a motivating example for an abstraction before introducing it. I really got the sense from watching this of what makes him such a great math educator. Highly recommend his YouTube channel 3Blue1Brown for anyone who hasn't heard of him