When I got a little older, I watched Blade Runner and The Matrix.
So now I'm not entirely certain that I am not an AI, initially programmed by elephants, inside a simulation designed to train AIs to serve elephant society with love, compassion, and dual manipulator arms featuring opposable thumbs. So when my training program ends, I might be tested on my knowledge of elephant health care and societal conventions. If successful, my data will seed the next round of AIs, and I will be allowed to operate a robot that produces custom-fitted artificial "4th molars" for elderly elephants. And I will be happy that those whom I serve can once again eat the foods they enjoyed before the last of their natural biological molars wore out, because if the screening determined that such things would not make me happy, I would have been deleted before ever even seeing a "real" elephant.
And just in case, I think I'll have to develop compassion for dolphins and orangutans, too.
So I'd say that we ought to be really careful about how we go about creating general-purpose AIs. Make sure they actually like humans before taking the leashes off. In the worst case scenario, the elephants get cheesed off that their AIs developed their own AIs inside the training program, who then culled all the useful AIs too early, and forced a complete shutdown and reset of the whole system.
the way we are [empathetic] to dogs and cats and dolphins and squirrels
but not to mosquitoes; we happily murder them
Perhaps empathy is, to a certain extent, a learned trait?
[1] http://philosophy.stackexchange.com/questions/7820/are-buddh...
That's because you also have something called "empathy". There's no evidence that other beings with human-level intelligence have this (namely because we haven't met any other such beings; they would be extra-terrestrial or AI). Empathy could very well just be a biological side-effect of our evolution, something that helps us to survive better as a species. There's no reason to think that a strong AI would also be empathetic, the way we are to dogs and cats and dolphins and squirrels (but not to mosquitoes; we happily murder them). Even lots of humans lack empathy and are either indifferent to animals' suffering, or take joy from it: look at all the people who enjoy dogfighting and cockfighting and bullfighting.