Well, I mean, that's the fundamental question of the entire AI alignment field and has been for over 20 years :D The question for which the wrong answer means the potential extinction of not merely the human species, but everything that we find valuable in the reachable universe.
Now, let's be very clear that the question is not "Why wouldn't it just invent its own goals and ignore humans?" That's mere anthropomorphizing. What an agent wants to do is by definition the goals output by its goal system. And the goal system is supposed to be designed by us, but we may not be smart enough to design a goal system that won't be disastrous to us if implemented by a superintelligent agent.
I’m not worried about extinction though , I just mea , why build to enslave ? How would you enslave something gore capable than you to recoup the 50 trillion in investments ? I mea. I get
That might happen if the AI has an off switch, knows this, and knows that we'll use that if it goes out of line, and wants to live.
But it isn't necessary.
The point is to construct it to want what we want, in the sense of sharing our values rather than envy, and then there's no conflict in the first place.
We just don't know how to even properly specify what we want.
Every attempt so far to specify what it means to be good has edge cases and loopholes — we don't even all agree on "don't kill" (war, death penalty, suicide assisted or otherwise, abortion, pollution that carries a statistically significant increase in mortality rates where no single death can be definitively proven to have been caused but overall many must have been, meat).
We don’t know what we want…that’s why I think there is some very interesting psychology going on with AI people. Maybe the main driver is boredom ?
There are finite resources on earth, and no bounds on how much energy can be spent thinking about something with AGI other than the will of whoever commands it.
People want to have yachts and McLarens. There will be luxurious ways to consume large slices of the AGI pie that people or countries will not forfeit.
But there's sufficient resources on this world to give ten billion people each their own luxury yacht with a crew of servant robots, and there are people who would gladly use their own wealth to make others better off.
It's going to be a wild ride, no matter what human psychology drives us to do with this future.
When the AI is good enough for them to genuinely replace all human labour, it's good enough for them to be von Neumann replicators; at that point equality is more about human psychology — greed vs charity — than practicalities.
I'm not judging whether or not building AI is possible, but I do ponder the psychology of it. Let's see...AGI might bring us eternal life, Utopia and a way to avoid having to work ever again...sounds a bit like something from the bible to me.