Re:Sam Altman: Why You Should Fear Machine Intelligence (2015)
by pixl97 on Wednesday November 22, 2023 @05:22PM
(#38386007)
You're mistaking terminal versus instrumental goals.
You're thinking of an AI with a terminal goal of "Kill all humans", and why would it have that goal in the first place.
But this is making a major error. You could give a superintelligent agent of "Buy gold at $400 an ounce using any means necessary, here is 1 million dollars". The AI could attempt to buy gold on the market at that price and fail and come up with the idea "Humans value gold too much, therefore removing humans will allow me to achieve my goals". Humans are just the ant hill in your front lawn keeping you from having perfect grass, you wipe the ants out with a second thought.
You're thinking of an AI with a terminal goal of "Kill all humans", and why would it have that goal in the first place.
But this is making a major error. You could give a superintelligent agent of "Buy gold at $400 an ounce using any means necessary, here is 1 million dollars". The AI could attempt to buy gold on the market at that price and fail and come up with the idea "Humans value gold too much, therefore removing humans will allow me to achieve my goals". Humans are just the ant hill in your front lawn keeping you from having perfect grass, you wipe the ants out with a second thought.