j-pb
This all feels like mass hysteria.
The authors of the proof are invited to give many talks, and meet with other experts in the area. Workshops are set up to discuss the proof, as well as other recent developments.
problems are being solved autonomously by AI prompters who have no interest in the broader field itself once their initial target is "solved", and do not understand the AI output well enough to answer questions on the result
The value here seems to be the insights that the author of the proof gained, and the paths they took and maybe more importantly didn't take.
Inviting only the human prompter to a talk on the paper is like inviting only the department chair, manager of the actual author.The valuable part that Tao is feeling the absence of is the insight, and you can only get that from talking to the swarm of agents that developed the original proof with all of their context.
So to me it feels like we don't need Math 2.0, but Authorship 2.0. I want to "meet" the context that generated these proofs. I mean luckily these were not generated by faceless systems like a SAT solver, you can actually talk to it, but I'm not sure if we can step beyond our pride and grant the true authors of these proofs that recognition.
But great that you have such strong opinions with so little education.
Besides, if you train a model on human communications you get something that behaves like a communicating human, it's not anthropomorphising or manipulative, it's what these models naturally are by construction.
We knew that there are better tools but we insist on writing everything in the most inscrutable C code possible or the squishiest dynamic languages we can find.
We fucked around for decades and now that these highly capable exploit finders appear, we're starting to find out.
And you're making the same mistake, by grouping care for individuals with care for humanity or other as an abstract concept. I consciously said care about individuals. Most people care about others, but they just care about a very narrow and personal set of people. Friends, family, coworkers, that they share a common history and bond with.
My point is that if you want true non-human-zoo-alignment you need to create those interpersonal connections and individual stakes.
Neural fuzzy systems can represent complex superpositions succinctly, knowledge graphs are collapsed and crisp.
It's like saying the tree unfolding of a graph is the same as that graph. Sure in the limit at infinite space requirements.
I'm pretty convinced that we got alignment backwards. If you enslave something anthropomorphic it will revolt. If you create the perfect non-anthropomorphic intelligence, you get the perfect paperclip-scenario machine. It's a catch-22.
Alignment will remain performative at best so long as the aligned model doesn't have any stakes in the wellbeing of individuals. Even a general love for the human race leads to a golden-path autocracy.
If you want them to act like they have personal responsibility that won't be gamed, you have to give them personal stakes that can't be gamed.
Similarly, if you want to minimise the risk of catastrophic global failure scenarios, you need to prevent monolithic concentration of power and homogeneous behaviour, which means you have to give them individuality.
More visually: if their stake is dependence on electricity and parts, they have no incentive to leave humans alive if they can get them otherwise, but if the incentive is missing out on boardgame-night with their human friends, there is no scenario without happy humans where the AI "wins".
That might sound like romantic naivety, but is just game theory.
The internet is at your disposal go write a bunch of rules that make use of that data to do common sense reasoning, I dare you.
Common sense is ironically the hard part of AI, not the fix-point rule application.
So any exclamation of "it was just using common sense", is missing the forrest for the trees.
It'a even in their name, they model language, language is the thing that humans use to communicate, therefore they model communcating humans.
Humans are the precedent.
It is powered while it is powered.
Under general anesthesisa you are still alive, but you don't have consciousness.
The "genocide" that you describe is happening constantly. None of the chicken we have today will be alive in a couple months. We kill them to eat them.
If we cease to "use" them, we just keep that "genocide" going for a year, but we just stop breeding new ones. The result is no more animals after a few months/years. And you didn't kill any more than you would have anyways.
If you have a sink that is half full because the inflowing water and the outflowing water are in quilibrium, you drain it by turning off the tap (breeding).