houzertuch
Imagine that neuralink tech starts to become normal. Inevitably, some parents will opt to have their child linked. Many children will be linked regardless because it is medically necessary. The linked children perform five times better than the linkless. They socialize by electronic telepathy, even though they only have threads in their motor cortex in the early days. Being linked also enables them to engage in wildly addictive and inappropriate behavior that had never been widely anticipated before. You become a parent in the midst of this. Do you imagine that you would opt out? Do you imagine the parents of yesterday thought they would opt out of allowing their children to do what children do now?
So you think that we should let all countries have whatever weapons they want under your logic. They will develop nukes and chemical weapons regardless of any international agreement that is established, so why even try? The obvious answer is to advance our own nuke technology as fast as possible so that we, the good guys, will lead where the arms race goes.
And AGI is a far greater existential threat than nuclear weapons. It is a greater existential threat than anything else, including global warming. It’s the biggest Pandora’s box in history. The idea of controlling or guiding its impact by having “the good guys” develop AGI first is the precipice of naivety. We lose nothing by trying to stop it. And we stand to gain more than we have ever gained from any coordinated effort. How easy or hard it might be is irrelevant, although it is much easier than basically anyone appreciates.
Can’t stop countries from doing it: yes, we can. A tin can dictatorship can’t do it. And besides, it’s a problem that depends a lot on academic work being published and shared among researchers. It’s a collaborative effort largely. Some rogue state trying to do this while avoiding inspections and inquiries by larger nations, and with no support from reading publicly published papers from other researchers, it would be tough. But with a large majority of anti-ai countries we could do a lot to prevent even covert attempts.
Humans are a technology. When there is other technology that does intelligent signal processing better than us, we will no longer proliferate. It’s amazing that we can see time and time again the arrival and departure of all kinds of technologies and yet we think we are immutable.
The reason why human history is filled with humans is because every time a country was defeated by another country or entity, the victorious entity was a group of humans. When machines are able to perform all the signal processing that we can, when they are smarter than us, this will no longer be true. The victorious entity will be less and less human each time. Eventually it will not be human at all. This is true not just in war but everywhere. In the global market. It’s just a simple and plain fact that cannot be disregarded.
As the people at OpenAI have rightly said, AGI is a compute-gated problem. It is a problem that can only be solved with very, very large amounts of compute.
The world has some total amount of computing power in terms of silicon based computation. For AGI to happen, there are two requirements: that this total be equal or greater than some theoretical threshold value for AGI and that the computing power is consolidated. So in layman’s terms, you have to have a lot of computers and they have to be connected in such a way as to efficiently share their compute. AGI will never come about if every individual computer were used to do research by separate entities but if all of those computers were connected into a single virtual computer, AGI might be discovered with them.
So clearly in order to prevent AGI, the best thing to do would be to address these two aspects. Prevent the total computing power of the world from growing and prevent computers from forming virtual meta-computers. Both of these tasks are in principle extremely easy.
Chip fabs are huge and expensive. Nobody is fabricating chips in their garage. This is just a hard fact. There aren’t that many fabs in the world and they are all highly susceptible to regulation. This isn’t prohibition of alcohol so please don’t confuse yourself. Nobody will be brewing chips in their cellar.
Let’s imagine that you could not regulate cloud computing. Let’s say the only way to prevent computers from offering their compute on a virtual market was to shut down the internet. This by default is the hardest way to solve the second aspect of the AGI problem and even it is very easy. This is because the internet is a large fragile collection of infrastructure that depends heavily on government cooperation. Nobody is going to string a fiber backbone for black market internet. ISPs cannot exist without regulatory approval.
If there were political awareness and motivation, and it was a global phenomenon, yes, it would be extremely easy to do what I’ve described. And since AGI is to the detriment of literally all people, it is not a far fetched scenario. And unlike alcohol in the United States, bootlegging would not be a problem. People in the USA think that banning anything whatsoever doesn’t work. It’s just fuzzy thinking, I can assure you.
I have made many predictions here on HN and they all have outlined that cloud computing would be the substrate from which AGI will spring. Now we see this announcement. There is a reason why OpenAI is making a deal like this with a very large cloud compute vendor: it’s because I’m right. And that means I’m probably also right in saying that we can stop this if we want to. You can’t just build a computer in your bad yard. And the internet is very fragile. Some simple regulation and global awareness and initiative could control what comes out of fabs and shut down the infrastructure necessary for cloud computing. It would be very easy relative to the size of the problem.
And this article makes it seem like California hasn’t done anything which is complete bullshit. They fail to mention the massive ADU initiative that effectively doubled the amount of housing in residential areas. Making things better for the developer and the landlord which will be a thousand times more effective than anything else. Even commercial developments use these ADUs.