> we probably should start thinking more about defensive AI
Isn’t this a paradox?
Parents often try to control who their children are going to be, and the children often rebel and become someone completely different. If it's a human-level general intelligence, you can't control who it decides to be.
if an AGI has any sense of self preservation, it will do whatever it has to do to not be turned off.
I mean that was my original premise supported by my article I posted. I go into detail on the conceptual methods for alignment and their fallacies.
When I state necessary, I don't imply the feasibility, it was in response to the question of the paradox.
Finally, the fact that AGI can not be aligned is also based on assumptions of its capabilities as well. If those capabilities don't manifest as we expect, that is really the only escape for the paradox.
Starting off by making the antivirus scanner sentient.
"I’m sorry but I couldn’t find any relevant information about the quote you mentioned. It seems like it’s not a well-known quote. Could you please provide more context or details about it?"
"The point of [AI alignment] is to ensure that the machines we create will be aligned with human values. And the reason we have to worry about it is that if we create machines that are more intelligent than we are, it's quite possible that those machines will have goals that are not aligned with our goals. In fact, they may not have any goals at all that we can understand or influence. This is the so-called 'provably unfriendly' scenario, where the machine has no motivation to do what we want, but is able to prevent us from interfering with its goals. The problem is that if we build machines that are provably unfriendly, then we will never be able to build machines that are 'provably friendly', because the unfriendly machines will always be able to prevent us from proving that they are friendly."
Then problem solved right? Super-AI will also be forced to take it slow if it wants its future self to be aligned with current self.
If alignment is possible but 10-100 years out for humans, then it is a problem.
That's making seemingly unfounded assumptions about both the AI's goals and its capabilities. It's also, I think, proceeding from a false premise — that it's impossible to align AI with "humanity" (which doesn't have a single set of goals/values to align to) doesn't mean it's impossible to align AI with an individual human or AGI.
Oh, wait...
Do dumb humans kill themselves solely on the basis that they can shift the ratio of smart humans higher?
AGI may have a goal of preserving the species without having the goal of preserving the self, this is not the case for humans.
Reminds me of nuclear weapons. Nobody is ever going to give those up again, because it would give them a disadvantage against those who do not give them up.
Destroy all fabs. Price of compute skyrockets. Huge international coalition engages in NATO-like agreement where AI research is considered an act of war. It would be extremely difficult for any entity to overcome that. Making and supplying a fab, creating and powering a server farm and then going through the arduous process of doing esoteric research all while being hunted by the largest international coalition in history. No. This is completely and 100% within the realm of possibility, it obeys physics and is similar to things we’ve already done — very out of the box but still not totally without precedent. This can be done. And it’s the only option we have.
The idea of defensive AI is frankly retarded. Have you played this out more than three moves? That leads absolutely nowhere. At best we attach ourselves as parasites to an AGI meta organism but it would only be delaying our extinction by a few years. Very quickly the insane, grotesque creates that emerge from the new world, shaped by natural selection among the machines into the most cunning, brutal and savage creatures that have ever existed, will wipe us out with or without some cobbled together defense strategy. Get fucking real dude. Be a part of the solution and stop brushing aside real solutions without any logical reason for doing so.
Sure, point me to the research papers and discourse on the solution you propose.
Alignment will be impossible. It is based on a premise that is a paradox itself. Furthermore, even if it were possible, there will be a hostile AI built on purpose because humanity is foolish enough to do it. Think military applications. I've written in detail about this topic FYI - https://dakara.substack.com/p/ai-singularity-the-hubris-trap
Stopping AI is also impossible. Nobody is going to agree to give up when somebody else out there will take the risk for potential advantage.
It seems we probably should start thinking more about defensive AI, as the above conditions don't seem resolvable. Of course, defensive AI might be futile as well. It is quite the dilemma.