With AI, the point is who uses it for what.
Think about it like a magic wand that makes your wishes come true. Would you really want everybody to have one? Who should and who shouldn't? Who gets to decide?
Humanity's strategy so far has been to pull marbles out of the bag as fast as we can. And that's been working out, because most marbles are white. But we're insanely lucky we haven't had a nuclear war. The USSR's early warning systems picked up 5 incoming american missiles in 1983. They didn't fire back because one of the officers on shift didn't believe the reports (why only 5 missiles?), and he declared the reports were fake. If a different person was on shift, there's a good chance nuclear war would have broken out because of a sensor glitch. And that's far from the only instance.
And now we're improving AI capabilities as fast as we can, despite it being pretty clearly another grey ball. AI has a lot of amazing uses. But why are we rushing so much? Doesn't seem wise.
https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alar...
> But why are we rushing so much?
Talking about these things with elaborate analogies only obscuring the point you already stumbled into: Humanity advances because humans are curious, driven, like to discover things, and like to push the boundaries.
Talking about humanity like one entity that operates together misses the point: There are billions of people in the world and some of them are going to discover new things.
There is no general counsel where humanity gets together and decides what humanity works on. There is no singular bag with a singular “humanity” pulling marbles out one at a time.
The analogy is too broken to be useful for the discussion.
This is true, but not all the time, and transparently not the case for the current leaders in AI development. Their motivator is money.
There are plenty of examples of this, e.g. when we discovered how to harness nuclear energy, we quickly realized that extreme measures needed to be taken to ensure we don’t collectively destroy ourselves.
You’re speaking of “we” as if the whole world was operating in unison, while there’s a war going on in Iran over nuclear disagreements.
The nuclear situation only reached stability after multiple countries fully developed it. I don’t see the parallels to halting AI.
Nuclear Non-Proliferation Treaty (1968)
Biological Weapons Convention (1972)
Asilomar Conference on Recombinant DNA (1975)
We've done this sort of thing plenty of times around specific, dangerous technologies.
Also the bio/DNA ones are quickly falling apart as the technology to do these kinds of things on a small scale are readily available. To the point where we're finding illegal wet labs in the US.
Nuclear was more of an exception because of many of difficulties in purifying stuff for weapons and its detectability.
AI is a much, much, much harder problem is it uses the already existing infrastructure as a base. Unless we regulate, monitor, and ban all private usage of technology AI can be developed at a small scale.
If there is a room full of people, and one person gets a gun, who would you pick to have it?
Yourself, obviously.
Maybe america will end up the only world power holding the secrets of LLMs? ... Oh no...
If nuclear weapons were as easy to build as AI models are to run, we would have all been dead decades ago.
There might have been. But the notes from the Manhattan project were leaked almost immediately to the russians by informants.
How is any of that good?
---
Simply put humans are at risk of any power scaling. We exist safely in a very tiny temperature and energy environment and the rest of the processes in the universe operate widely out of that strict safe area.
It's also a catch-22. There are large rocks in space that will eventually end us unless we do something about that. But having the power to do something about that gives us the same power to also end everyone on the earth.
> Nazi Germany undertook several research programs relating to nuclear technology, including nuclear weapons and nuclear reactors, before and during World War II. [...] The first effort started in April 1939, just months after the discovery of nuclear fission in Berlin in December 1938, but ended shortly ahead of the September 1939 German invasion of Poland, for which many German physicists were drafted into the Wehrmacht. A second effort under the administrative purview of the Wehrmacht's Heereswaffenamt began on September 1, 1939, the day of the invasion of Poland. The program eventually expanded into three main efforts: Uranmaschine (nuclear reactor) development, uranium and heavy water production, and uranium isotope separation. Eventually, the German military determined that nuclear fission would not contribute significantly to the war, and in January 1942 the Heereswaffenamt turned the program over to the Reich Research Council (Reichsforschungsrat) while continuing to fund the activity.
> The program was split up among nine major institutes where the directors dominated research and set their own objectives. Subsequently, the number of scientists working on applied nuclear fission began to diminish as many researchers applied their talents to more pressing wartime demands.
> [...] Heisenberg said that in 1939, the physicists at this second meeting concluded that "in principle atomic bombs could be made, but they also emphasized that it would take a number of years",[24] certainly not less than five. He also said that he "didn't report it to the Führer until two weeks later and very casually because I did not want the Führer to get so interested that he would order great efforts immediately to make the atomic bomb. Speer felt it was better that the whole thing should be dropped and the Führer also reacted that way."
> [...] Speer states that the project to develop the atom bomb was scuttled in the autumn of 1942. Though the scientific solution was there, it would have taken all of Germany's production resources to produce a bomb, and then no sooner than 1947.[39] Development did continue with a "uranium motor" for the navy and development of a German cyclotron. However, by the summer of 1943, Speer released the remaining 1200 metric tons of uranium stock for the production of solid-core ammunition.
There are ways of calling this "weren't working on a nuclear bomb", but it seems to me like "they wanted to, were moving in that direction, and would have done it if they had the resources and weren't distracted by shorter-term considerations".
That sounds very much like "they weren't really working on it". They explored the options and decided not to.
Whatever you want to call it, that's what I was referring to.
Like all forms of evil once invited into your house it does not leave at your say-so, another profoundly negative feature of nuclear warfare is that now we've invented it unilateral disarmament will likely always be strategically impossible.
Still seems a moral cop-out, and while true, the timeline would certainly have changed, if not the inevitability, if key figures such as himself had opted out.
I'm not arguing about the consequences or responsibility. I'm asking which humans would be capable of stopping the development of AI at this point in history.
If we're talking game theory, a better model is the Stag Hunt [1]. At some point the equilibrium strategy is to cooperate.
It’s difficult to understand where this all is coming from because it’s so strangely disconnected from history. It’s not even consistent with current world events. The US is in a war with Iran over nuclear proliferation while also worrying about North Korea’s nuclear program. China and the US are in a trade war and Taiwan is under threat due to semiconductor capabilities (among other things, obviously). Open weight models allow anyone in any country to do AI work on their laptop. The top story on HN yesterday was a small team of academics making some cool discoveries on improving LLMs without frontier lab level compute.
How anyone can see all of this, in addition to decades of history of attempted nuclear diplomacy and wars, and posit that the whole problem is easily solved by simple agreements is confusing to me. The confidence with which these alternatives are being proposed and the unwillingness to acknowledge second order effects (like AI researchers moving to a different country and continuing to work) is baffling.
But yeah, there are a lot of comments from genie-rebottlers that are not very grounded in reality. They're functionally equivalent to saying "hey, everyone in the world, let's not actually use electricity after all" in 1905.
This looks a lot like the “if I didn’t do $badthing someone else would have” line of reasoning.