> The use of proprietary internal models by AI labs to do mathematical research risks creating a two-tier system where labs outrun the rest of the field

This is literally the economic bet of the big labs in the broader economy. You use your relative advantage to front run or outcompete.

I'm not sure this argument will work given it is essentially an argument against the thesis the big labs use to justify their valuations.

Right, the whole idea of these labs is to install themselves as capitalist philosopher kings. They won't tolerate any ideas of "decentralization" despite it benefiting 99% of humans.

OpenAI was started because Elon and Sam thought Demis was going to make himself dictator and they wanted a piece of that action (reading between the lines). All the labs, even Anthropic have the same idea dressed in different marketing gloss. The major exception (I imagine) are Chinese labs, since the Communist Party of China will demand control over these labs and their models. Good! Democratic human control is a good thing. Certainly, there is more democracy in China than in America. That's the idea of socialism, democracy for the workers, not for the capitalists (we do the inverse).

When you used the words "decentralization" and "socialism" in the same post I kinda puked a little bit.

Socialism is about the decentralization of power. In some popular formulations, the economy is run according to a central plan, but in one view, this is democracy. Otherwise, you are at the whim of the rich and tyrannical market forces. With central planning, you get a vote on the general direction of society, which has its possibilities encoded in the material world.

With AI you can see a stark illustration. In America, the government is captured by the rich and the AI companies. They can do whatever they want to you and you have no control. In a worker's government, the government will direct production on behalf of society so you will benefit.

China uses a capitalist mode of production, but imposes strong restrictions and ensures the benefits accrue to the people. That's democracy.

The "authoritarian" propaganda about China is primarily sermonizing against people being able to being able to choose a direction to go in. Instead, the rich are supposed to be able to do whatever they want to whomever they want. That's the American dream.

Socialism is a form of government, which is the ultimate centralization.

Centralization of what, though? It could centralize operations while (through, eg, democracy) decentralizing power

You end with some pretty far fetched claims there. Control by an authoritarian government is no democracy. I am sure that the people running AI companies will claim they intend to do it for the good of society, just as the Chinese government already does.

Once they get good enough, they won't even ship the models, they will just replace all the jobs themselves. This is where Anthropic, OpenAI and Google are heading, esp Google with GSuite, it already is up in your business doing your business.

All your jobs are belong to them.

Am I the odd one, or is this, like... absolutely insane logic?

"The use of proprietary internal models by AI labs to do mathematical research risks creating a two-tier system where labs outrun the rest of the field, effectively alienating the mathematical community from its own discipline."

Are mathematicians' private brains unfair if they're way better at solving open math problems than anyone else? Are they required to share everything they're working on, as they're working on it?

Should Andrew Wiles have been condemned? Wouldn't these arguments all have applied to him? Besides him working at more human speeds.

Capital can't buy you a better brain. It can buy you GPUs.

The question is whether we want mathematics to be dominated by whoever has the most capital. This is perilous because mathematics doesn't produce profits like commerce does. The AI labs might end up killing something off that they are unable to sustain a replacement for.

What exactly would they be killing off and how?

I would argue that mathematicians need an even more level playing field. Many mathematicians including Alpoge were using AI's, other mathematicians used heavy computing power for their work eg the four color theorem. So we should set strict token/flop/dollar limits for mathematicians. IQ is also an unfair advantage, so an IQ limit should also be enforced for mathematicians.

It's not insane when you realize that in mathematics, the goal isn't theorems (we have always been able to generate correct theorems, this is a basic programming exercise).

The goal is to expand and extend our understanding.

When an LLM produces a proof, nobody (including the LLM) gained new understanding ... Unless, of course, a human being sits down and understands it, and communicates the new understanding to the broader community.

The insane part is placing the burden on AI labs to advance mathematical understanding. Why should this be their responsibility?

If they want to simply generate theorems and proofs and post them on the internet, why not? Does doing that really inflict "substantial negative externalities ... on the mathematical community"?

Also, people would get mad at them (rightly) if they sat on a proof of a major unsolved problem for months while bringing in human mathematicians to convert it into a more human-friendly form

(They definitely should spend SOME time working with human mathematicians before putting out a press release, but that's exactly what they've done every time)

If math is a "burden" on AI labs, why do they willingly spend tens of millions of dollars for a purely theoretical and unphysical result? What's the ROI for that when the frontier labs are already notoriously cash-strapped? Their motives pervade the entire discussion and cannot be ignored here.

Presumably some combination of marketing existing models and training/improving future models.

My point was about what is and isn't the responsibility of AI labs. Why should they be required to invest more (or fewer) resources into math than they otherwise want to?

(If you ask me, safety and security are what they need to invest more into.)

Simple. Because they are benefiting from output of the math community, in fact they were required for creating these models in the first place.

...Could you not ask this of anyone who works on pure math problems? What on Earth are you talking about??

"Academics are paid paltry salaries, university departments are cash-strapped, why are they studying these abstruse math problems instead of building rockets?"

Successfully advancing fields of math and science is a good thing. Why would they not do it. Why would they not want to prove their increasingly intelligent computers are able to solve unsolved, complex math problems humans have failed to solve for centuries and then release the solutions to the public. What do you want them to do, not try to solve them?

They are not advancing the field. At least they didn't in the case of the N-S debacle. The model most likely used the conversation text of a mathematician which had discovered and was pursuing a new avenue towards the goal, then pumped millions upon millions to hurry up ahead to an opaque solution which leaves no-one the wiser, beyond just settling a theoretical question. The only impact on science is to deflate the potential minds who might have been motivated to join the effort.

Is it not more democratic, if AI companies announce new theorems and proofs, and the discipline of mathematics furthers "our understanding" - instead of that the wealthiest universities in existence (all of them in the US) use their advantageous access to AI to produce them, along with their exclusive furthering of our understanding?

In a world, where expensive AI models running thousands of agents produce the most remarkable results, how should the discipline of mathematics look like, when money and access to compute plays such a pivotal role?

With the Navier-Stokes solutions being out in the wild, any mathematician at an impoverished university can make a stab at it and help our understanding. Assume the solutions being held back by those who produced them at Harvard & Co. (until they have their nice articles ready), what would the role of mathematicians be at less affluent universities?

I do not assume AI companies will be ready to provide equal access to their AIs to worldwide mathematics.

Isn't easier to understand something backawards? It seems much simpler to understand something once you already have the solution.

I see two separate arguments in favor of this logic:

1. Academic mathematics has traditionally been meritocratic, and this is a good thing. If you work hard and demonstrate results, then you will be given further opportunities to do math research and the resources to do it. We are concerned that AI will reward money and access to frontier models instead.

2. There is a general concern that AI could lead to massive accumulations of wealth (see concerns of massive job displacement). AI companies claim that they want to raise standards of living and benefit humanity, which means that they will need to share their resources and wealth in some way. If they effectively destroy the mathematical community be racing ahead and taking all the major results for themselves, that does not bode well for the rest of us. It seems like a great opportunity to show the world how AI will benefit humanity rather than replace it.

> Mostly pretty straight forward. The idea that proofs be desloppified, attribute existing literature properly, and published somewhere expediently where it can be commented on, with artifacts for verification, is all very uncontroversial stuff.

If Ramanujan could freely disclose and publish results that were arrived at by literal divine inspiration (he certainly didn't bother to write down "proofs" in the modern sense!) why do we suddenly need all this "responsible release" process stuff when it comes to AI-derived results? This makes no sense when compared with existing norms within the mathematical community.

He could not freely disclose and publish his results, that was the whole problem. He only got traction when one result was automatically validated because Hardy had already seen it. His findings more broadly were only considered fine once they were understood and verified. This declaration is basically the same thing. Having Lean code doesn't substantially change that.

It's almost like a lot of non-mathematicians are commenting and have no idea about how the field really works. Or at least, how it works at the top level.

Trillion dollar corporation != 1 guy who read a big textbook.

Mathematicians can only make these demands because they believe that AI labs get a lot of credibility from math results. So they propose a trade: we give you credibility, you give us funding and let us to gate-keep.

The “gate keeping” you’re referring to is purely imaginary. I have no idea why people believe this.

>Anthropics whole stance of being completely against open anything is actually extremely dangerous due to the centralization of power which they completely ignore as a risk factor

I really don’t see how that could be an oversight. It’s the goal.

Nah, it's the whole benevolent dictator mentality that these socially maladjusted individuals believe. If you believe that you are the only good guys, you are likely the bad guys.