To be blunt, this seems incredibly uninteresting to me. I enjoy learning mathematics, sure, but I just don't find much inherent meaning in reading a textbook or a paper. The meaning comes from the taking those ideas and applying them to my own problems, be it a direct proof of a conjecture or coming up with the right framework or tools for those conjectures. But, of course, in this future, those proofs and frameworks are already in the textbook. So what's the point? If someone cared about these answers in the first place, they probably could have found the right prompt to extract it from this phantom textbook anyways.
You could argue for there being work still like marginal improvements and applying the returned proof to other scenarios as happened in this case, but as above, what is really there to do if this is already in the phantom textbook somewhere and you just need to prompt better? The mathematicians in this case added to the exposition of the proof, but why wouldn't the phantom textbook already have good enough exposition in the first place?
I think my complete dismissal of the value of things like extending the proofs from an LLM or improving exposition is too strong -- there is value in both of them, and likely will always be -- but it would still represent a sharp change in what a mathematician does that I don't think I am excited for. I also don't think this phantom textbook is contained even in the weights of whatever internal model was used here just yet (especially since as some of the mathematicians in the article pointed out, a disproof here did not need to build any new grand theories), but it really does seem to me it eventually will be, and I can't help but find the crawl towards that point somewhat discouraging.
Who cares if it is God's book or the machine's Xeroxed copy?
Any scientific basis for this claim?
Pre-conception is unlikely to be really possible outside some esoteric circles. While in the womb there could be some limited experiences that get ingrained in the mind as memories, but I don't think that's the topic here.
"The Book" is more interesting to me if I am the one coming up with the ideas to fill it in. Maybe this is a bit egotistical, but I'd like to think it is allowed to have a desire that you, personally, are contributing to something in a meaningful way. Like, if you are on a sports team, it'd be more fun to win a game if you were on the field than if you were benched, and I think that's okay. And ultimately I don't find dredging for proofs from an LLM particularly meaningful, nor do I see it as a particularly personal contribution, as anybody else could have done the exact same thing with the same prompt.
This isn't to say I wouldn't love to read the proofs in "The Book" for problems I care about, I just think I'd eventually get bored of only reading. And so its hard to be enthusiastic when this book is being built through an LLM.
This is a good analogy for AI work displacement. Probably would resonate with some of the college students who boo'ed Eric Schmidt.
Technology in general (smartphones, social media, search) even without AI is creating this feeling, as it shrinks the world and makes it less mysterious.
It's worse than boredom it's more like nihilism.
Then when you strip purpose and meaning from a human you get something very bad, despondency being the best case outcome.
It will be a transition, for sure - there would no longer be meaning in “winning the game” in a capitalistic or scientific sense. Anything you want to produce or learn, the AI could already produce or has already learned. Now you have to do it just for the love of the process.
I have a musician friend who likes to say that good artists overwhelmingly make art for their own benefit. Not to advance the world or blow people’s minds, but because something inside of them needs to come out, and art is how they express it. And that part of us isn’t going to go away.
Shifting from “human calculators” to machines for arithmetic is also hard to argue against.
I think what makes the AI transition difficult is it impacts a wide range of high-value activities that would have been implicitly assumed to always remain human.
I do have great trouble seeing how a pile of matrices is ever going to be capable of innovation. Maybe with sufficient entropy and scale, it will… The day that becomes practical will be a turning point in history.
Economically, goods and services are often priced based on labor/“value added” aspects. Lawyers’ fees aren’t driven by paper costs! If AI takes a huge bite out of intellectual labor, the future could become very different…
BTW, your book description reminds me of the 2025 movie “A.I”. I thought it was quite good.
You admit this possibility so I'm not arguing with you, but it seems far more plausible to me that we can build something better than the brain.
In the limit we can just grow brains and put them in computers anyway, then the debate is moot. That's a really hard problem but of course not physically impossible.
I'm also afraid of a world where AI completely replaces human mathematicians, but if we remain collaborators, then that's a world I can still feel excited about.
Along with all the rest of what humans find meaningful and fulfilling.
It may be the beginning of thinking, but to many who view things on a longer timeline. It starts to look like it will breakdown the frameworks of which are required to get to that position. Otherwise, you just end up retreading explored ground. This removing the joy of discovery from any humans hand/mind.
Perhaps your name-calling is not actually as logically grounded as you think. It definitely seems to depend on unfounded leaps.
This technology is solving interesting math/physics problems for us, which is completely different.
Humanity is having those discussions, heck you are in one RIGHT NOW not some Hollywood future.
What is coming of those discussions is the ownership class balks at the idea of raising their taxes (see recent interview with bezos), and therefore balks at the idea that you or I should have any value beyond what we produce... And if AI can replace you or I, well how do we survive if we can't produce in a technological society?
Money is valuable only as it changes hands for goods/services, and if you want to get rich, on top of having/producing/controlling something everybody desires, you also need as many people as possible to have money to give you in exchange for a piece of that something.
i wonder if this is physically/mathematically impossible: the mere act of living involves processing energy, and therefore doing work :)
And there is a lot of energy to be processed in this Universe before the heat death...
Mind you, there are places in the universe that we have no way of knowing ever existed... The non-obserable universe if you will. For when physicists talk of the observable universe, it is only the fraction we have any chance of receiving data/light/radiation of/from
This "any" shines like a thermonuclear fireball.
In the (probably unlikely) event that AI use results in a post-scarcity economy in which there's no need to work to survive, a lot of people wouldn't regret sentiments like the ones in question.
On the contrary, it would mean they could work on whatever they please, including potentially standing on the shoulders of giants - the AIs - and seeing even further.
If we actually worked to create a society that work for the benefit of all its members, there would be a lot less reason to worry about developments like these. Much of the worry arises because for various reasons - none of them really good ones - we've ceded control of these developments to the people least suited to manage it.
To a society that provides a livelihood to all humans, equally?
For, I would love to hear how we get from here to there during an era with the largest wealth disparity ever seen in human history. (Yes, it's worse than the robber Baron era of US history). For I have yet to see any signs that the capital/ownership class has any intentions other than vacuuming up even more wealth and power for themselves. And that anathema to your desired outcome.
History is full of examples of situations like this being corrected, at least to an extent. If we learn from those, we can do even better next time around.
Btw, the inequality you mention is far worse in the US than Europe. Here's one source that covers this: https://wid.world/es/news-article/why-is-europe-more-equal-t...
This demonstrates a point that should be obvious, that better societal choices can produce better outcomes.
Even taking a purely Kantian interpretation that would scale this beyond mathematicians - and that itself is a logical leap! - making a universal law out of "a discovery can be beautiful regardless of whether created by humans or AI" is is much more specific than the straw extrapolation you've created.
"Let's try to think posts through."
If 20% more medical knowledge would save more lives long term, there are actually people, probably some browsing this website right now, maybe the person you're responding to, that actually think killing people up to the expected number of lives saved is justified.
I would personally call that evil, but it is thought through.
Not so many years from now, some of them will surpass you. A few years after that all (that survive to that point) will surpass you.
Does that terrify you just as much?
A child is a living, breathing, growing, and changing conscious entity. It is the natural order for the young to supplant the old, no matter what the politicians and billionaires desire.
"AI" - terrifies anyone who understands the pact our society rests upon: that labor is valued and can be exchanged for goods and services to survive. Thereby enabling a person to support their families without having to do everything themselves.
If AI replaced a noticeable fraction of society, destroying their capacity for work. That threatens and ultimately blows up this compact between working class and capital class... With it, the foundations of a modern technological society.... It may sound like hyperbole, or some fantastical prediction. But really it is basic economics, like econ 101... And personally the last few years have terrified me, not because of AI directly, but because how ignorantly blind many smart and tech savvy people are... You are marching us to collapse with a smile on your face...
Moreover, truth be told, I don't really see myself doing any less math and requiring less from my skills. At least from the moment I've begun incorporating LLMs into my research workflow to now, the demand I've had from my own skills has only grown. At least in an era prior to Lean formalization.
This is just an application of the philosophy "automate yourself out of a job every 6 months"- I've been doing that for a long time, and the outcome is generally a more interesting job.
A dedicated engineer is always looking to automate themselves out of existence, so that they can move on to the next thing to automate. Ongoing repetitive work is less engineering and more akin to toiling on a line.
The answer is that we simply need to decouple the "right to exist" from "worth."
You should have the right to exist and explore the world simply because you're human, not because you can use your skills to provide some sort of transactional value to someone else. Deprogramming so many people is going to be hard...
Let's start with the first practical step: how do you dethrone the psychopaths in charge of the world who own about everything on Earth and have all the world's lethal force in their pockets?
Perhaps it is time for life to be considered intrinsically valuable, instead of being "worthy" only based on output or capability. Disability, animal and environmental advocates have been fighting for this for a long time. Not too long ago women and minorities were in the same boat. Even now, there are many advocating and fighting for a return to the dark old days.
> Along with all the rest of what humans find meaningful and fulfilling.
Some humans. Many are content to enjoy simply existing, and the beauty of life and the universe around us. Just like many non-scientists today enjoy and benefit from the work of scientists, tomorrow too many will enjoy learning from, and applying the coming advancements and leaps in many fields.
And those of a scientist or other research-type mindset? No doubt they will contribute meaningfully by studying the frontier, noting what remains unanswered, and then advancing the frontier, just like researchers do today; just because scientists in the past solved many questions doesn't mean that there aren't any questions to answer today.
IMHO, AI means that the frontier expands faster, not that it is obliterated. Even AI cannot overcome the laws and limitations of physics/universe: even Dyson spheres only capture the energy of one star, thus setting a limit on the amount of compute, and thereby a limit on intelligence. And we are a loooong way from a Dyson sphere.
PS: I think you're being unfairly downvoted. Your question is not invalid and deserves responses, not downvotes.
Ah, the proverbial silver spoon. Sadly, I never had that luxury. If you look through my comments, you'll notice I'm more at the get-off-my-lawn point.
Also, what happened? Real world wear you down and turn you cynical? It is possible to be hopeful and cynical at the same time. This tech is something new we're seeing: the future is as yet unwritten. r/LocalLLaMa works well, so there's hope even if corporate ai goes kaput.
My generation has been lucky to see a few new things, though we certainly live in interesting times. Moon Landings. Berlin Wall fall. Moore's Law. EU (I have the old coinage to serve as a reminder). Space Shuttles. China and India integrating with the world. Cellphones. The Internet. Digital Photos. Linux. Solar. 3D-printing. Smartphones. Tablets. Bitcoin. EVs. Mars rovers. Asteroid visits. Internet from space. FTTH. MRNA. Gene Therapy. MRI. Ultrasound. Wi-Fi. Mesh Wi-Fi. Reusable Rockets. Cubesats. Selfies from space. Drones. LoRa/LoraWAN. Maglev HSR. And now AI, real AI. Chinese-like Whale Language.
There's hope for the future yet. You can help make it happen right. But only if you leave the cynicism at the door. Can't give up - it's our kids' futures at stake.
What about Ukraine holding Russia back, and now looks like it might actually win? What about the most recent additions to NATO. Hungary's regime change? Canada's save? EU's pivot to arming itself, and quickly?
Buds of green, yeah?
Can you please expand on how you do so?
[1] https://en.wikipedia.org/wiki/Connections_(British_TV_series...
the hype is where the money is, as is always -- marketing & porn. Both touched heavily by AI already.
That a raw LLM hallucinates?
That we never see all the mistakes and dead ends a complex system using AI hits?
Does it even matter if its accuracy rate across all its experiments is < 100% if it can run trillions of experiments in the same time a human could run 1?
We don't see many of the failed attempts of Human Researchers. Why? Because it doesn't matter.
What amazing here is that it shows our society can make discoveries faster in the post LLM world. Thats incredible.
Your "critique" of how it happened. Not so much.
I'm sorry to nitpick, but isn't an unconscious idea an oxymoron?
Unless you’re defining an idea as something that only exists in your conscious awareness?
I bet your stupid ideas also taught you a valuable lesson and you learned at least something from the experience, maybe your next idea won’t be so dumb, and those 100 watt hours weren’t actually wasted (though it may feel like they were). Compered to a failed LLM experiment, where all those billions of billions of computations are completely wasted. the model knows exactly as much after a failed experiment as it did going into it. Those Megawatt hours were simply wasted, turned into heat energy, paid for by raising the power bills of the of the datacenter’s neighbors.
Not my post, but I think point 1 is stronger than 2.
That's not necessarily true. If our only counterfactual to investing resources in project A were to invest them in some other project B, then, yes, the conclusion above follows. But often people just consume the resources.
(In the end, the goal of all economic activity is consumption. We invest resources so that we can consume more later. If there's no good enough project around, might as well consume more now.)
When you consider the amount of computation which went into this discovery it is less impressive. Like if you spend a lot of fuel you can travel really fast, much faster then a bicyclist. Similarly Go-engines can beat the best humans at go, but they spend several orders of magnitude more energy to do so.
Mathematicians prove or disprove conjectures all the time and use orders of less energy to do so. Using LLMs is kind of just throwing money at the problem and hoping it works. In this case it did. But this is not the most efficient way to do this, and it won‘t scale.
From what I can find AlphaGo Zero runs at ~400 watts, while a human brain uses ~20. So really only about 1 order of magnitude difference.
Of course, training costs are a different question entirely.
Currently we can live with it because someone can review that work. Soon we wont be able.
Always, always always, the problem with research and development is leadership, not insufficient supportive technology. It is a political problem, there is absolutely, positively no shortage of technologies to support research. Your optimism is totally misplaced. The NSF funding cuts have negatively impacted math more than AI has benefitted it. And guess who supports the administration that cut NSF funding? The people who ousted the PhDs from OpenAI.
You are right to point out that the ones who fully own and pilot the machines all belong to the “fuck science and humanity as a whole” group. So the likely outcomes don’t look good.
Echoes the early promise of the internet vs the eventual state and consequences of it, although seemingly primed for far more dire and deeply penetrating consequences.
No interest in human advancement, just attribution.
What I’m saying is that the ultimate goal of those in power are not these sorts of altruistic or even scientific pursuits, and that the massive labor disruption and hyper concentration of power in the hands of those who are proving time and again that advancement of science and benefiting the whole of humanity are actually antithetical to their goals is likely a bad thing.
Most homeless people have smartphones, and consistent access to food and clean water.
Your average 'poor person' in America has HVAC. An unimaginable luxury in the EU
Eh, don't be silly. In the places where the summer is hot enough (or, more precise, where it used to be hot enough), I have seen plenty of AC units on shabby buildings, even on old Commie apartment blocs in Romania.
AC is not that expensive.
Lmao, did HN just glitch out and start showing me Pieter Levels' tweets?
Southeast or central US has considerably higher wet bulb temperatures than Europe does in summer. Without HVAC, there’s a good chunk of the year where it’s too hot to get much done.
That's true. But. Maybe you've seen the Oppenheimer movie, there is a moment where Oppenheimer shakes Teller's hand, basically after the guy ruins Oppenheimer's life in a completely immature betrayal. That's what people are angry about, the academy community is Oppenheimer's wife asking, why the fuck did you shake his hand?
At least regarding leadership and funding, I don't know if it's a matter of likely or unlikely outcomes. It's just facts: these guys are collaborators. The commenter might very well have zero graduate students starting next year. What pisses me off is the utter obliviousness that STEM people have about how deeply political their work is.
And perhaps this is the real reckoning for the mathematics community. Not the possibility that AI is going to replace their jobs, it's not going to do that. But that having these intensely myopic and disagreeable personalities mean that basically zero leadership skills have been nurtured in the mathematics community. You cannot name a single politician who is a mathematician. You have to be elected to have power in this country, it's that simple, there are way more billionaires than there are presidents! Leadership is far more scarce. So that's why these disputes matter, and while it's great that people engage on Hacker News about it, it's intensely disappointing that "reduced science funding is really bad" gets downvoted.
That is a result of Hacker News's emphasis on this very 2010s view that it wants to be a place where the math nerds gather (in @dang's words) - he doesn't get that the quality of the discourse was caused by great leadership at many political and academic levels. Nobody credits how much better leaders were during Y Combinator's biggest success stories, or how much we overvalue the intellectual powers of math because it makes money as opposed to enlightening our view of the world.
I can: https://en.wikipedia.org/wiki/C%C3%A9dric_Villani
( and of course Wikipedia has a list :) https://en.wikipedia.org/wiki/List_of_mathematician-politici... )
For example, this library here for deep learning is 100% ai generated and far beyond my technical capabilities.
For example, if you're in fundamental science (or generally a fan of reductionism), it for sure would be nice to understand the universe instead of just having access to an AI that can comprehend it. But to the majority of the population it only matters that someone (or something) understands it enough to make it useful to others.
AI is going to both help and hinder this process though. At the end of the day, mathematics is mostly a social process at this point. The goal is not raw number of theorems proven, it’s how proving theorems affects the working operational models of mathematicians. Only a rare few new theorems in mathematics nowadays have direct real world applicability.
If AI produced legitimate theoretical breakthroughs at a pace mathematicians are unable to absorb, then the impact will be neutral to negative.
It seems like if AIs can prove and index a huge number of (largely uninteresting to humans) things there might be sort of "parallel cultures"? Big results are most valuable to humans and AIs both (most context efficient!), but a very large number of less general but still non-obvious results might be an effective approach to solving problems?
Has this ever been different?
Math is abstract, rightfully so. It does not have to have direct applicability. Understanding builds over time and applications eventually follow. Number theory used to be a fringe "pure" theory field without applications for the longest time. If we'd only be interested in (and thus fund) what has direct applicability then society would be much worse off.
Side note: I recall my high school class mates rolling their eyes in every math class with "when will I ever need this in my life?" never asking the same question about PE or history or art classes. Now they struggle with their tax return and are routinely getting screwed over by loan sharks. But make no mistake, they can be proud of their A for hitting the goal 5 out of 5 times during soccer in PE class.
I am no mathematician and very naïve about this, but in a world that is rapidly becoming extremely calculation and network dependent that sounds hard to believe.
> If AI produced legitimate theoretical breakthroughs at a pace mathematicians are unable to absorb, then the impact will be neutral to negative.
I think the idea here is that all mathematicians will just be using AI for their future work so they don’t really have to absorb it as long as it’s in the training data.
> I am no mathematician and very naïve about this, but in a world that is rapidly becoming extremely calculation and network dependent that sounds hard to believe.
I am a mathematician. It is true. The key is we're talking about new theorems, and direct, current real world applicability. Some theorems that have no applicability now may in the future, as theory often precedes applications by a long way and the usefulness is likely to come from other things built on top of the new maths, and a lot of pure maths will never have direct real world applications but contributes to our overall understanding.
On the other hand, there are many applied mathematicians and theorists from other fields that mine new maths for applications to their fields. But they are almost always not the ones that come up with the new math.
Historically, of course, mathematics was always driven by the need to explain things. Many of the mathematicians from the 17th and 18th centuries were physicists (or, less commonly, engineers). But for the last hundred years or so that really hasn’t been the case.
That is not what the mathematicians are saying. I don't have the knowledge to evaluate this myself, but a number of mathematicians - for example, in the SP - are saying it goes further than that - they really do introduce novel ideas. Of course everything is based on and inspired by some previous work, but that is true of all human mathematics as well.
LLMs that have been trained through reinforcement learning on mathematics are NOT simply token predictors. Only base models can be accurately described that way. They have learned how to do mathematics. They have learned to do coding. Its really amazing we're three years into instruct models and such a large part of Hacker News still does not understand the most basic facts about this field.
"All" a model is doing is predicting the next words, based on the statistical distribution of words it has seen similar to the ones read/produced so far.
We push a model towards a particular set of distributions through context. If I ask a model "What is the capital of France?", there is a non-zero chance it goes down the dad joke answer of "The letter F". The far more likely option is "Paris", because the joke appears much less often in training material, but if I wanted to be absolutely sure of getting a consistent geography answer I'd address that with additional context. We can add context via prompts, RAG, agents, skills and so on.
However, when training a model, we select the material. We could show it a lot more geography information (or dad jokes!), and skew the statistical distribution in the direction we wanted. We could also decide to design the system prompt towards the direction we prefer - which the user would interpret as "the model" - and so nudge the context model-wide. We can also construct the interaction to iterate on context with a specific framing and call it "reasoning".
In this specific example, you could therefore solve the problem by a) training skewed towards mathematical papers, which likely degrades performance in general and likely for the specific case too, b) train the user to provide better context/prompts for mathematical work, shifting the workload to them which feels very "a la 2024", c) publish agents and skills that are tailored to mathematics work (very "a la 2026"), d) tweak the system prompt for when the model is doing mathematics work, which the user would see as "the model" doing the change, but you and I might look under the hood and say that is in the harness or a specific type of prompt, or e) add "reasoning" execution that is set to focus on mathematical formatting, or f) a mixture of the above.
Right now we're probably looking at agents and skills. I think over time we're going to see smaller models targets towards domains with a mixture of all of it, where some of this sits at user configurable levels, and some is "baked in" via training, system prompts and execution modes, but from a user perspective it's all just "the model".
And by opening the door to LLM-generated results, you'll see greater and greater amounts without any hope of ever navigating this field again without machine help.
It's a little like a software project which more and more gets extended by a AI agents with less and less review by human software engineers and in the end the complexity and spaghetti design are so incomprehensible by humans that the maintenance requires an AI agent. The risk is that math as a whole (the field itself) will experience that effect.
Say we achieve interstellar travel, but nobody actually knows how it works.
Or we cure cancer, but the "cure" requires a microrobotic implant, and it runs as a blackbox AI, and only the other AIs can make one, and there's no guarantee they will know how to make one tomorrow.
Or we solve global warming but it requires giant cooling machines running 24/7 and again, nobody knows how it works, but with the added bonus that the planet is cooked if they ever stop working.
It's hard to describe the feeling of seeing intelligence being delegated increasingly to AI. If that's not a pivotal moment, a revolution, I don't know what is.
This has always been true. There was a time where someone had to teach farming to others and that information had to spread and be passed down. Eventually, farmers became better than hunter-gatherers and they became known as hunters. The information on what was safe to gather for civilisation got passed down as 'safe to eat on the hunt' because the farmers were farming. The civilisation collectively "forgets" foraged foods as that knowledge becomes niche.
Does that mean we got dumber?
Looks like you're pretty sure of that. Every time I see argument like this delivered with confidence I wonder how is it different from, say, digital calculators. Or better yet, books - Greek philosophers moaned that young people will stop understanding anything and just check books when they want to know anything.
Knowing the history of the humankind is what makes me pretty sure of that.
The extent of misery and destruction is directly proportional to the level of technological advancements, and I don't like the idea of sacrificing millions of lives in the name of the figurative HVAC, smartphone and other benefits of civilization. Or billions in the name of whatever benefits the next VC money stake should bring.
> I wonder how is it different from, say, digital calculators.
Did a single digital calculator ever stop any war, or liquidate a psychopath who orders people to go kill and die?
Benefits of civilization eliminated most of that + increased quality of life dramatically.
Ten thousand years ago (around 8000 BCE), the global human population was estimated to have been roughly 5 million people. This is significantly smaller than the current population of just Poland (about 36 million).
In absolute numbers there might be more now, even if the percent is smaller. It is difficult to compare this things without having a specific place in mind.
The fact that humankind grew from 5M to 8.3B, while dramatically improving longevity and quality of life speaks volumes. Multiply life quality × population × life duration, not only "misery and destruction" is not the case, but you could rather see powers of positive technology influence.
If you had a magic button that turned off all those "benefits of civilization", millions would die. If you managed to drag agriculture down with the rest, the death toll would be in the billions.
I don't understand how you can possibly think you "know history" without recognizing that technological progress has taken us from constant warfare to such a state of abundance that war is actually rare and noteworthy in much of the world.
Let's try to have an actual argument. How many people, in absolute numbers, were affected by that constant warfare of past, which past exactly do you mean, and how many people were/are affected by "rare" wars of modern history?
80 million people killed or maimed with arrows, swords and catapults over centuries and 80 million killed or maimed with fruits of industrial revolution over 6 years of WWII are very different figures.
Is it better to have lived as an individual one of these fictional cohorts? Is it better for the group in the same or different one?
Is it better to live and suffer than to not live?
I think the answers are obvious.
If only 4 people die violent deaths out of a total population of 5, that’s an extremely violent population to be a part of.
If 8 people die violent deaths out of a total population of 100,000. That’s a much more peaceful population to be a part of, despite the greater number of absolute deaths.
Orders of magnitude more than in prehistoric days.
By statistics of war, poverty rates etc this is trivially false. I think you are really, really underestimating how hard life was pre-industrial revolution.
The intriguing part is that we could get objectively good outcomes, but at a cost of being dependent on the machines. So it's not that you couldn't actually unplug Skynet, it's that if you did civilization would collapse (or whatever) because Skynet stops doing its thing.
I'm not sure that gets us to a better place overall, but I doubt we could resist the temptation.
True, but it is possible to assemble a team of people that does, with backup for each person. There's also teachers and written knowledge to educate new team members. That's what makes it resilient.
I think that's a very different situation from what's decribed.
The idea being that once a toolchain becomes sufficiently complex if you ever have to bootstrap it again for whatever reason you won't be able to speedrun the process the way you might naively expect. I think modern chip production likely already reached this point several decades ago. As evidence I'll point out that China only recently achieved EUV and remains several nodes behind despite directing an obscene amount of resources towards the initiative.
1/ No one knows how even small components work, because their inner working mechanism is too hard to understand by human mind
2/ The whole society is run (in intelligence sense) by alien minds
When you let the machines do it, and don't care about moving it towards human domain (i.e. meatspace), you're done.
I think there will be regulation that requires some users of AI to provide an explanation upon request. For instance, banks could be required to "explain" why you didn't get that loan. What if the decision is based on a credit score that includes some AI prediction that ultimately relies on the entire training corpus?
The bank can give you a list of factors that play into the decision but they may not be able to explain deterministically why a very similar customer did get that loan. At that point I think we're going to resort to statistics that prove a lack of bias against certain protected characteristics, but that's not really an explanation, is it?
I think we will never get useful and complete explanations for everything that AI does. Society will just accept some explanation-like thing or proxy and move on.
If they understood it 100%, what clarification is needed?
Your dog will never understand calculus or why Fourier transforms are interesting. There's almost certainly topics that are beyond human comprehension that an advanced artificial or alien intelligence can easily handle.
Why is it necessary to continue to increase complexity when we get better intelligence? Can't we find more simple solutions? Or at least more explainable.
What your describing is already how a lot of science, technology, and engineering works!
https://youtu.be/pfNS2kWf5cY?si=SH6_QC0bCspV-ngz
There are comments that truly reveal a future horrifying and true. Few of them. But I count yours among them.
But I’d argue also that airplanes already achieve this complexity to some degree as well as microprocessors.
I mean, microprocessors have been on the "impossible to bootstrap from scratch in a short period of time" for 20 years already.
In case of AI we have a better chance to understand what it is doing through chain of thought and explainability. Nature never gave us that..
That’s not “solving it”, that’s putting a bandaid on it. Solving it would mean correcting the underlying issue to the point it’s no longer a problem which requires maintenance.
Managing symptoms is not curing the disease.
Green energy and transport technology is now at the point where people save the world and get rich trying, just as fast as they can build the factories.
Food's climate impact is harder, because the problem isn't technical, it's convincing people to give up beef (and other things, but mostly beef).
* quantum mechanics and general relativity are famously difficult to get to grips with
The book doesn't deviate from what you have envision, or the future you envision doesn't deviate from the book, I may say.
I do think we will need to find a way to get away from publishing papers. But I thought that before the AI came along and made mediocre papers something you can produce in a day. The academic system seems utterly incapable of self-correcting on this point though. We haven't even managed to get rid of for-profit publishers. So how this all will go down is anybodies guess right now.
I am also a little worried about what it means for your training as a junior PhD. Often you would try and solve a problem your advisor thinks is doable that they assign to you as a learning exercise. It may be more and more difficult to find problems that a junior PhD can solve but that AI can not. Tim Gowers has written about that here: https://gowers.wordpress.com/2026/05/08/a-recent-experience-...
This is a very important point, especially when the output is from a non-deterministic random walk with some unknown probability distribution.
They do the opposite by locking the results the produce within the slop presentation that needs more AI to comprehend.
I attended a conference on AI for maths and open science a few weeks ago, and was struck by just how many examples of AI-supported solutions there already are. Virtually every speaker had an example of either their own use of (often the frontier) AI models in solving a problem that was previously too hard (for various definitions of hard).
I wrote up a few notes [1], and most of the speaker videos are available via the conference website [2].
[1] https://scholarlyfutures.substack.com/p/ai-and-the-practical...
Many of my colleagues and I have been experimenting with LLMs in our research process. I've had pretty great success, though fairly rarely do they solve my entire research question outright like this. Usually, I end up with a back and forth process of refinements and questions on my end until eventually the idea comes apparent. Not unlike my traditional research refinement process, just better. Of course, I don't have access to the model they're using =) .
Nevertheless, one thing that struck me in this writeup, was the lack of attribution in the quoted final response from the model. In a field like math, where most research is posted publicly and is available, attribution of prior results is both social credit and how we find/build abstractions and concentrate attention. The human-edited paper naturally contains this. I dug through the chain-of-thought publication and did actually find (a few of) them. If people working on these LLMs are reading, it's very important to me that these are contained in the actual model output.
One more note: the comments on articles like these on HN and otherwise are usually pretty negative / downcast. There's great reason for that, what with how these companies market themselves and how proponents of the technology conduct themselves on social media. Moreover, I personally cannot feel anything other than disgust seeing these models displace talented creatives whose work they're trained on (often to the detriment of quality). But, for scientists, I find that these tools address the problem of the exploding complexity barrier in the frontier. Every day, it grows harder and harder to contain a mental map of recent relevant progress by simple virtue of the amount being produced. I cannot help but be very optimistic about the ambition mathematicians of this era will be able to scale to. There still remain lots of problems in current era tools and their usage though.