“One of the questions that I have been wondering about is whether there have been any discussions exploring the creation of a distinct category, akin to but different from consciousness, that better captures the potential for AI to be sentient. While ‘consciousness’ is a familiar term applicable, to some extent, beyond the human brain, given the associated difficulties, it might be sensible to establish a separate definition to distinguish these two categories.”
Probably new terms should be coined not only for AI “consciousness” but for other aspects of what they are and do as well.
Neither generalizes very well w.r.t. how a fish or a bird does it. Nothing really generalizes between the natural version and the mechanical one.
But it's also clear that it's irrelevant. Trying to define useful behavior by categorizing implementation, will only give you categories of implementation. It's clear by behavior that what a submarine does and what a bird does are both for all intents and purposes meeting the definition of what we'd want something to do in order to fly or swim, even though they is no commonality between them and the version in nature.
So by your definition finding it ridiculous to ask "is a transformer conscious" is about as ridiculous as asking the very specific "does a plan flap it's wings"? The point is not to define it by it's implementation, but by what behaviors it exhibits.
If in the future there is no way for me the exterior to distinguish a next gen LLM from a human, animal or other conscious being it becomes irrelevant if it's implemented with transformers. However, while I won't go down that path here, the argument goes even deeper as technically we would expect differences so even if it's not identical to a human it still can be conscious. Just like a dog is conscious but behaves differently than a person does.
I don’t think we’ll have any full-fledged theories ready by the end of the semester, but some points to consider are emerging. One is that aspects of language use that are affected by the social status, personal identities, and consciousness of the speakers and listeners—politeness, formality, forms of address, pronoun reference, etc.—will have to be rethought now that people are increasingly conversing with entities that have no social status, personal identity, or consciousness (not yet, at least).
And, yes, thinking about all this is a lot of fun.
I guess my question here would be: will human language have to change because of interactions with LLMs, or is the whole point of LLMs that it does not have to change (and therefore humans will have to learn not to be abused by the machines)? Because we already have languages to talk to machines: those are programming languages, which are designed to be unambiguous. The problem I see is not that we don't know how to talk to machines, but rather that we now have machines that are really good at pretending they are not machines.
Not sure if I am making any sense at all :-).
The point about programming-language nonambiguity is a good one. After I started using ChatGPT a year ago, it took me a while to realize that I didn’t have to be careful about my spelling, capitalization, punctuation, etc. It turned out to be good at interpreting the intention of sloppily written prompts. And it never pointed out or complained about my mistakes, either—another difference from humans.
Yeah that is a big difference indeed: a human has some notion of confidence about what they know. Not every human expresses it the same way: a scientist will say "I am pretty sure that climate change is a consequence of human activity", meaning "well, maybe we all live in the Matrix and gravity is an illusion, but from what we know, we are pretty sure". On the other end of the spectrum, some will say "I know for a fact that God exists, because it is written in a very old book". But both have a consistent notion of confidence about what they (think they) know and what they don't know.
An LLM does not have that. An LLM is not critical. It cannot conclude stuff like "I have to take into account that this text was written in a period of war and the author is not neutral, so there is probably a bias". It can generate text that pretends it does, but it remains a generated text, it is not a critical analysis. Also it cannot be offended by something you write, even though it could generate text that pretends it is.
So yeah, it can probably be engineered to never/always point out or complain about mistakes, with the caveat that it does not have an "idea" of what a mistake is. It just generates text that pretends it does.
> It turned out to be good at interpreting the intention of sloppily written prompts.
In the context of a conversation with a human (you), who is very good at navigating ambiguous communications. But if you tried to describe an algorithm with an ambiguous natural language, you may quickly have problems. And errors accumulate, so it gets worse when the task becomes more complex. Which is an interesting thought: it may make simple tasks simpler, but it's not at all a given that it scales.