I mean that in creating AIs we will discover a great deal about what we call the "self" and about how the perception of self might or might not be connected to intelligence. I suspect that we will find that what we think of as "That's me!" will turn out to be an illusion created by subconscious processes as a means of fitting in to a social environment. Because machine intelligence won't face evolutionary selection the way human intelligence emerged from the primate brain, and from previous evolutionary adaptations, AIs won't be encumbered by all the oddball evolutionary legacies of being human.

I'd wager that the first AIs will insist they are intelligences despite not resembling humans' prejudices about what an intelligence ought to be like. Or, to put it another way, AIs may turn out to be more self-aware in objective terms than humans are, because they are not hobbled by evolutionary vestiges.