The fact that the word "conscious" is a synonym for several concepts is entirely irrelevant to this conversation.

The concept that we can't yet capture formally is the kind of sentience that living, awake humans surely have, most animals probably have, and rocks definitely don't have. The limits of this are harder to pin down, as, again, we lack a deep formal understanding.

But we do know exactly what we'd like to formally model.

> "But we do know exactly what we'd like to formally model"

Do we really? I thought it was a deeply intractable problem for cognitive theorists and other such experts.

Haven't you ever encountered a challenge that you at first thought was straight-forward and obvious to solve, but after digging in find it was unexpectedly nuanced and difficult?

> "The limits of this are harder to pin down, as, again, we lack a deep formal understanding."

Doesn't that contradict?

If you were to try pinning down precisely what you mean by "the kind of sentience that living, awake humans surely have, most animals probably have, and rocks definitely don't have", I suspect it will not be nearly as easy as you think.

How do you know rocks don't have it, for instance? How do you know you do, for that matter?

> How do you know rocks don't have it, for instance? How do you know you do, for that matter?

These are both part of the (incomplete) definition I gave you.

There have been problems like this before in the history of science - we have concepts that we intuitively understand, and want to give a formal definition for. This can take a long time of careful study - such as coming up with a complete definition for "insect". Other times it can indeed turn out that the concept was ill-defined - there is no possible formal definition for the concept "fish" that doesn't appeal to the human mind (the various organisms we think of as fish are too diverse genetically to find a satisfactory one-to-one mapping with any group of animals, even if you exclude a small set of them, like the aquatic mammals).

Ok. I'm responding to the statement that these are "relatively clear concepts". AI will by definition never have "the kind of sentience that living, awake humans have" (and I do acknowledge that this is only a working definition) because the goalposts will continue to move according to whim in the absence of a more formal definition. I remember when one prevailing formal test of self-awareness was "independently claiming to have self-awareness". There were caveats, of course, but that goalpost has evidently moved.

We do know what it means for ourselves to be sentient.

We do not know how to translate it to anything outside of ourselves. That's why it's not readily formalizable.

Yes! Descartes helped us to understand our own sentience. I can accept the proposition that because I think, I am.

Thoughts are not necessary for existence.

"Because I think, I am" implies more that existence is necessary for thoughts, which is true. If you didn't exist, you couldn't think.

"Because I think, I am" just implies that I exist because I think. This implies that thoughts are a necessity for existence. I am saying that is not the case and thoughts are not a necessity for existence. What you inferred, although true, is not the correct implication of the statement.

I was reading it more similarly to "Since I think, I am" which would be closer to "I think therefore I am" than to "I am because I think", though perhaps that's giving it too much slack, or perhaps its coming from a native language where word order is more flexible than in English.

Got it.

Natural language is vague and allows for many possible interpretations of short phrases. The one that is more applicable to Descartes quote is something along the lines of "me thinking proves that I exist". This, of course, says nothing about thoughts being a necessity for existence.

Got it. Now, it makes sense.

Yeah, sorry I missed all of this. I agree that thinking is almost certainly not necessary to exist. I probably could have written it more as "Because I think, I can be certain that I exist"