Basically, because science can not define terms like "consciousness", because the term represents what we don't know and we can not define what we don't know.
What are examples of things it might say that would cause you to conclude it must be "conscious", by your definition?
Also, you seem to have ruled out animals like dogs or chimps. I've never heard of one them them describing their own consciousness. Are you saying they aren't conscious if they can't describe it? (should we not be concerned if an animal is in pain, if it can't describe pain to your satisfaction?)
I'm not saying animals aren't conscious. I'm saying they can't prove it. Remember, if A implies B, then not-A does not imply not-B. I'm saying that if a creature can describe its own consciousness without having learned of the concept elsewhere, that's good evidence that it's conscious. If it can't, that doesn't mean it's not.
Not at all. I remember having qualia-like questions at 5 or 6. Particularly, "do you see red the same way or do we just have the same name for that color?" An AI could definitely ask these kinds of questions without prompting.
I understand your "can't prove it" regarding animals, but if your only way of "proving it" doesn't work for most examples, I would continue to say the concept is one that is not defined in a meaningful way, at least not meaningful when it comes to any sort of scientific analysis or even being able to say whether an entity other than yourself has it.
I agree that it remains fairly poorly defined, but I don't think it's correct to say that you can't even analyze it from the outside. There are many circumstances where you can't, but you can at least glean a little bit of information in some cases.
The point, though, was how do we tell if something else is. You haven't given us much in order to show one way or another if either an animal or a human-created thing is. So I'm saying the concept is so weakly defined as to be essentially meaningless.
Also, you say that because humans have somehow developed the ability to talk about something, it must exist. What exactly have they said? That there is this thing that they can't define objectively but they swear it exists because it seems like it does? And that proves that it exists? Do you not see how circular that is?
How would humans have arrived at the idea of qualia if they didn't experience it? What's your proposed alternate explanation for this?
I've seen qualia described as "qualities or feelings, like redness or pain, as considered independently of their effects on behavior and from whatever physical circumstances give rise to them."
I go into a good bit of detail about how such a thing is unsurprising in this thing I wrote (8 years ago). http://www.karmatics.com/docs/violet.html
The thing is, you have this intuitive feeling that something (say, redness) is real. Fine. But the external behavior you are describing as "proof"? I would consider that beyond weak. There is nothing at all surprising about an entity that can communicate, expressing that there is a certain something that it is having trouble communicating directly.
> I can't help but have a lingering feeling that there must be more to the concept of "subjective experience" than "arbitrary details of the internal implementation of an algorithm that don't affect the results," but alas, I can't come up with words to describe whatever else there might be.
I sympathize, but disagree. It's quite clear to me that the sensation of perceiving things, whether it's redness or sweetness or pain, has something about it that is not just arbitrary details of how my brain works. There is a me which is qualitatively different from the rest of the universe in some way. No, I can't really describe it very well, but that doesn't make it any less real.
You do have me wondering, though, if maybe not everyone has this. Take aphantasia, for example. Some people can see things in their mind, and some can't. For a long time, neither group realized that the other even existed. Those who could see things just assumed everyone could, and those who couldn't assumed everyone couldn't. Maybe some subset of humanity consists of p-zombies, and they only know of these concepts because they've heard of them from others.
I don't think that's exactly what I'm saying. Maybe I am saying that if you can't define it in a reasonably rigorous way, that it doesn't make a lot of sense to claim that it absolutely must exist. There is a difference.
Of course this is real to you. But I am saying it is not real from the perspective of science. For instance, it isn't something that can have value that natural selection could have selected for.
It is, by nature, subjective. And that basically means "from a particular perspective." If you view it from a different perspective -- for instance, the external objective view that is appropriate in a scientific context -- it disappears.
Asking "is some entity X conscious?" is sort of like asking "last December 6, what were the chances of there being a big earthquake in Indonesia the following day?" The only way to answer a "what are the chances?" type question is from the perspective of someone with limited knowledge. From the point of view of us, today, we don't have limited knowledge, we know the earthquake happened. So the question doesn't make sense.
Likewise with consciousness. You can't analyze it from a perspective other than that of the entity in question. It doesn't make sense.
There are plenty of other ways of defining consciousness that make it more accessible to science. But you might find it difficult to make sure it isn't inclusive of things that you don't consider conscious, like a self driving car or even your phone.