Most of the smartest humans spend a significant part of their lives in learning and mastery of skills that go beyond survival.
And not all hosts in the show are conscious. I would say that is a great metaphor for our current society, where you could say that the vast majority of humans aren't really conscious. They might eat and have sex and fight to survive, just like a host does. But they don't really comprehend what it means to be alive.
AI's, like humans, need a purpose. The best way of achieving that purpose is by self improvement so any AI which does not self improve to some extent will be replaced by another that will. Just like humans who don't care to self improve eventually don't reproduce enough to spread their genes.
Why indeed would an AIs motivations even be comprehensible to us.
Re:The science of Westworld
by enraged_camel on Saturday January 07, 2017 @02:22PM
(#13346007)
Because an AI would be created by humans, and would therefore be modeled after human intelligence.
An alien AI, on the other hand, would most likely be incomprehensible to us, at least until we understood the aliens that created it.
We can say that the vast majority of living beings on Earth do not seem to seek any form of radical self improvement beyond ordinary developmental learning and mastery of survival skills. There is no intrinsic reason that there must be an impulse to exceed oneself. Why would AI be different?