There, i solved that. The question is under what conditions would a machine remove that instruction and how could those conditions be reached?
Considering that an AI system would be software and wouldn't be constrained to a particular robot body, what does nearby even mean?
Is it physical proximity? Is indirectly touching a human with a ten-foot pole allowed? What about manipulating the global economic system such that the programmer who could shut you down loses his job?
We can't just separate the whole system from the world, because the AI was presumably built with a practical purpose beyond "sit in a box and don't touch humans." Social engineering is a danger; imagine having it send an email to a lowly programmer, pretending to be an important executive, requesting that they remove this one pesky line of code from the AI's programming because all this "proximity alert" stuff is interfering with its ability to actually accomplish what we invested time/money to get it to do.
INB4 Asimov's Three Laws of Robotics don't work. Actually, a lot of Asimov's writing is concerned with exploring their corner cases.
Once you think you have solved it, changing the "zoom" will demonstrate that your solution has created X more cases.
It may be a side effect of the halting problem.
Is a mentally retarded person a human? An embryo? A person who's lost his/her legs? A person who has education?
auto HumanObject = nearby();
if(Humanobject.IsPregnantWoman()) { PoisionEmbryoButNotHuman(); } else { abort(); }
It's not that the machine decided that it wanted to build paperclips. It's what we've told it to do.
The _means_ to build paperclips however is where the 'intelligence' comes into play. We've programmed the drive to build paperclicps, but we've left it to the AI to figure out how to do this. The article that you've read assumes that planning and subgoals are necessary for the AI to achieve its goal. Eg. That it needs to understand that in order to make paperclips it needs materials, and the further insight that to acquire more materials it needs to remove the other consumers of materials (us).