I get the scenario where people use "AI" for their purposes. That is of course a very real scenario. But the question I raises was in relation to the OP's point about "AI" taking over the World and exterminating humans.

I think this [1] article makes a good case for a lot of AI futurism just being a new draping over though complexes created for old religious debates, so the AGI in the doomer view is basically the devil: it will conquer the world because that's what the devil does, and it will do it by seducing people using clever lies.

I'm not saying that recreating religious tropes is what people set out to do, more that as they walked trough the bare space of possibilities they found these well worn paths of logic and simply settled on what felt like convincing lines of reasoning, not seeing that it felt convincing before it was familiar.

[1] https://www.vox.com/the-highlight/23779413/silicon-valleys-a...

You're mistaking terminal versus instrumental goals.

You're thinking of an AI with a terminal goal of "Kill all humans", and why would it have that goal in the first place.

But this is making a major error. You could give a superintelligent agent of "Buy gold at $400 an ounce using any means necessary, here is 1 million dollars". The AI could attempt to buy gold on the market at that price and fail and come up with the idea "Humans value gold too much, therefore removing humans will allow me to achieve my goals". Humans are just the ant hill in your front lawn keeping you from having perfect grass, you wipe the ants out with a second thought.