Right, but as long as the phrases you parrot appear to continue a conversation that I usefully model as produced by (another?) consciousness, I pragmatically (see Peirce) don't care what the actual mechanism that produced them may be. (an analogy: when using computers we almost always care that a protocol is followed, often care that a specific function is computed, sometimes care about which algorithm computes it, and almost never care about the exact implementation) If you were to be a giant lookup table or a Searlean room instead of a conscious being, well, that's on you.
> what if I am a P-zombie dreaming "I" am conscious? — not Zhuangzi
see also https://xkcd.com/810/
But when that's all I care about, I don't ask that question. To me, the question of "consciousness" matters in, and only in, scenarios where the inner world matters, which is why I use that for my definition of "consciousness":
1. If I uploaded my brain, would that be a fork of my consciousness that would survive the death of my physical body? Tautologically only if it's "conscious".
2. Are the AI we make and use suffering a fate worse than the actual slaves in the Caribbean who invented the term "zombie" out of a fear that their masters would force them to keep working after death? Only if they're "conscious", and even then possibly not (suffering is an extra question to be asked once consciousness is determined).
I'd kinda like to get my brain uploaded, but there's no point if it's not a continuation of my consciousness.
I'd like to use a super-smart AI to solve my problems, but not if doing so involves creating minds that exist only for a few seconds before being destroyed.
— "Make me one with everything," said the monk to the hot dog vendor
(Vendor gives hot dog to monk, monk pays)
— "..." waits the monk
— "How much was that hot dog, anyway? I gave you a 20" says the monk
— "Change," replies the vendor, "is an illusion."
Sure, but that same chasm of difference which means we can't guess at that (despite it being so innate to many of us), means we could very well accidentally cause suffering or induce rage even when we think we're being nice.
Seeing my own phrasing echoed back, I notice an ambiguity: "the same kind" can be either "all the same things" or "it is kinda close".
The former is provably different between humans: aphantasia, gender dysphoria, sexuality (including the absence), ASD, the conservative/liberal divide, that some people describe having religious experiences and others don't, amusia, … there's so many ways to be different, I'd be surprised if any of us is entirely "normal".
I meant the latter. And the only positive evidence I have for that, is that other people were writing about it before I was born… which doesn't actually tell you that I have it, as for all you know I might be a P-zombie that's just parroting phrases it once read, I'm the only one who can be sure that there's an "I" in this body of mine.