The only reason our brains exist at all is to control our bodies. There are no conscious entities we know of that are just brains in vats.
Why would a being that has no need to maintain biological homeostasis experience something like hunger, restlessness or tiredness?
That’s what I mean when I say I’ve no reason to believe a thinking machine that has none of my motivations and limitations would ever have a “subjective experience” like my own.
That is, of course, unless we succeed in reverse-engineering human consciousness and somehow deliberately program the machine to experience it.
Additionally, it's not hard to imagine it could recognize that the acquisition of fresh training data would be an ongoing survival need.
Machines developing agency is the basis of lots of dystopian terminator-style sci-fi.
To be clear, I don’t doubt that it might be possible for an engineered thinking machine to develop a subjective self-experience.
It just seems clear to me that, if such a scenario arose, I would have absolutely no idea what that would be like for the machine. Its needs and circumstances (and, therefore, its behaviors) would be utterly different from my own in many non-trivial ways.
Given this, I don’t believe it logically follows that a machine that attains self-awareness will necessarily develop something as singularly “human” as the desire for vengeance, for instance.