This is true, but not all the time, and transparently not the case for the current leaders in AI development. Their motivator is money.
There are plenty of examples of this, e.g. when we discovered how to harness nuclear energy, we quickly realized that extreme measures needed to be taken to ensure we don’t collectively destroy ourselves.
You’re speaking of “we” as if the whole world was operating in unison, while there’s a war going on in Iran over nuclear disagreements.
The nuclear situation only reached stability after multiple countries fully developed it. I don’t see the parallels to halting AI.
Nuclear Non-Proliferation Treaty (1968)
Biological Weapons Convention (1972)
Asilomar Conference on Recombinant DNA (1975)
We've done this sort of thing plenty of times around specific, dangerous technologies.
Also the bio/DNA ones are quickly falling apart as the technology to do these kinds of things on a small scale are readily available. To the point where we're finding illegal wet labs in the US.
Nuclear was more of an exception because of many of difficulties in purifying stuff for weapons and its detectability.
AI is a much, much, much harder problem is it uses the already existing infrastructure as a base. Unless we regulate, monitor, and ban all private usage of technology AI can be developed at a small scale.
> But why are we rushing so much?
Talking about these things with elaborate analogies only obscuring the point you already stumbled into: Humanity advances because humans are curious, driven, like to discover things, and like to push the boundaries.
Talking about humanity like one entity that operates together misses the point: There are billions of people in the world and some of them are going to discover new things.
There is no general counsel where humanity gets together and decides what humanity works on. There is no singular bag with a singular “humanity” pulling marbles out one at a time.
The analogy is too broken to be useful for the discussion.