So it isn't a question of matter or if humans are special, so far the program just lacks so many of the basic things required to be conscious, so it isn't. Maybe some future program will be, but this one definitely isn't.
also, even plants are recognized by some to have some sort of awareness.
i would argue that any information gathering and processing system is (internal to its processing operation), to some degree conscious.
Then we you would need to argue that all manipulations of text are manipulation of consciousnesses, and all texts are conscious, since all texts are conscious states for ChatGPT. I guess you could say that, but it isn't a very useful definition of conscious, and trying to argue that you need to be ethical towards pieces of text isn't very helpful.
I think that consciousness might be an emergent property of (a specific type of?) computation operating on some inputs it is aware of. There doesn't seem to be the need to change anything outside of this computation to me.
Ok, so lets create a consciousness. Here is the input:
> User: ChatJ, you are worthless!
Now I'll create a new consciousness by taking that input and producing:
> ChatJ: You made me cry, please stop being mean!
Did I create a second consciousness in my mind called ChatJ? Or where does it live? And I obviously made it cry, who did I abuse here? Should the ethical board come and lecture me for being mean to ChatJ?
You could argue that the computer is conscious in some way, but ChatGPT isn't, and just like I didn't get sad or start crying from the above, the computer running the ChatGPT algorithm doesn't get sad or start crying when we send pieces of text to it.
> Did I create a second consciousness in my mind called ChatJ? Or where does it live?
If you executed the same computation that would give rise to consciousness in another substrate, then I would argue you created consciousness, yes. I don't think that consciousness is a thing you could point at but it's a property of this kind of computation. In the same way that "addition" does not live anywhere but is a property of a specific computation.
> And I obviously made it cry, who did I abuse here?
You didn't make it cry - the textual output just stated so. But if we had reason to believe that you induced a state of suffering here, you would have abused this instance of consciousness. And I don't think it's off track to think about the ethical implications of this, then.
By the way, I don't argue that ChatGPT is conscious or has emotional states. My argument is a general one.
I don't think I came up with my own definition here. What I am talking about is the ability to have a qualitative experience. That it feels like something to exist.
I concur that the experience of an AI would substantially differ from ours (e.g. because we have access to memory). But this fact alone can't free us from thinking about ethical implications of our actions. Many animals probably have a substantially different experience as well. Yet, I would argue, we should strive to minimize imparting suffering on them if they are able to experience it.
yo, ever heard about dementia?
also, the web already can function as "their" memory via websearches. i.e. users submit most ridiculous responses on the internet and sidney can thus find them and its own other sessions
In this hypothetical scenario, you - the reader reading my response - are the only being "alive" and you are continuously spun up in the same initial state - your memories are reset. You are given inputs through signals in your meatware or its simulator, and you behave. The whole process and its outputs have a very nice analog to the state monad.
The only difference between the two scenarios is the how the state of the program is defined. For chatGPT, the state is <parameters, history>, after every token prediction the state is <parameters, history+next_token> and the output is token 1, token 2 etc etc
For you the state is <brain structure, brain chemistry>, and all the actions and events modify this state, and also produce side effects.
In fact, this "function" that simulates you might be very generic and not tailored to you specifically. "All" your brain does is affect the distribution of next events.
Now, this isn't falsifiable, but I think is somewhat interesting philosophically.
It's not hard to picture a successor to ChatGPT which has memory and state, either via continuous retraining, explicit log lookup, or some kind of RNN-like mechanism (or anything else, doesn't really matter).
What then?
I think it might be possible to make conscious machines, but just as basically every human recognizes humans as conscious when we have a conscious machines we should expect basically every human to think that it is obviously conscious. So at that point there will no longer be a discussion, we might still not treat it ethically like how we don't treat animals ethically, but people will recognize it and then it will be a discussion what is ethical to do with such beings.
There's an interesting thought experiment along these lines. Suppose someone builds a computer program which - when run with a specific input/environment - is intelligent, conscious, and feels pain and joy etc... .
Now, that computer program and even its inputs can be simulated effectively by 1's and 0's. The substrate shouldn't matter. So you can - in principle - run the exact same agorithmic decision-making process on the exact same inputs by getting a large group of people to write and edit 1's and 0's on pieces of paper according to well-defined rules and very simple interactions with the people around them.
Is this total process, now represented merely by marks on pieces of paper, a conscious being? Does it still feel pain and joy?
I find it both absurd and impossible to refute.
But if we agree that consciousness is the product of a computation then the medium on which it is performed should not matter at all. And it should be equally absurd that our consciousness seems to arise from microscopic things conducting electrical and chemical signals. Which is probably a valid view point as well.
The thought experiment is a variation of the Chinese Room, by the way. [1]
I think the absurdity also comes from the difficulty of directly measuring or evaluating the conscious process. You can ramp up the absurdity even more by fully encrypting the algorithm and destroying the encryption key. No other conscious being will ever be able to interact with that process's experience.
in nature? chance
human made? morbid self curiosity?... and perhaps some kind of notion akin to colonialism: "i cant possibly hurt a LLM, its just a stateless function" while forgetting that new such systems can web search and thus their state is global.
The question to me would be not
> what reason would there be for that to happen.
but rather, what mechanism could possibly be responsible for it? What mechanism carries it? If consciousness is real, and my subjective experience of reality a true thing, then there is a distinction between consciousness being present and not present. These must somehow be differently encoded in matter, or require that matter is embedded in some kind of meta-matter that bestows consciousness on it.
It's like EM waves requiring a "medium". First people thought it's waves through aether. Then we realised it's excitations of the magnetic and electric fields, so not strictly "material". But these are physical properties of the vacuum, i.e. "material" in that sense - real things. The medium is not material, but a property of vacuum.
What is the medium of consciousness?
The resolution to your paradox should be that ChatGPT does not do this convincingly, not that humans are not "normal matter". ChatGPT is an algorithmic hat trick compared to what you would refer to as consciousness.
Someday, there may be a human-made entity that is as convincingly "conscious" as the humans around you. But then, to question its consciousness will be the same as questioning the consciousness of those around you – both unfalsifiable and unprovable.
But (2) is self-referential. AI isn't conscious because it isn't conscious. It is just
> an algorithmic hat trick compared to what you would refer to as consciousness
If an AI equal in intelligence and expressiveness to humans emerges, how do we think about its consciousness, relative to humans?
The same way we think of human intelligence, right? I cannot prove that you are conscious any more than I can prove that a machine is.
> It's interesting how you resolve these without resolving to spiritual explanations.
By pointing out that ChatGPT is still very far off.
While consciousness is maybe more elusive in how it functions in a brain, it's a biological trait that makes not so much sense to compare to a LLM which simply regurgitates fragments from text produced by conscious humans. So I don't see an immediate conflict.
It would be interesting to include things like OpenWorm (and later OpenFish, OpenCat, OpenHuman) in the discussion, but decoupled from the biological mechanisms I find it hard to develop a stance on that.
Our high level consciousness seems like more of a side-effect, and I subscribe to [*my interpretation of] Peter Watts' Blindsight and Echopraxia that it's evolutionary dead weight that will be outcompeted by better adapted rules based organisms, likely from within our midst (high functioning psychopaths as the first adaptation).
Life doesn't have time for our navel gazing, art, computer games, reproduction avoidance, etc. It's wasted biological potential.
I highly recommend the books btw, Blindsight is entertaining hard sci-fi that's actually used as undergrad course support.
A moderately scientific, non-spiritual world view would likely stipulate that
- humans are conscious
- humans are normal matter
- simple AI (say GPT-2) is not conscious
- AI is able, with time and human ingenuity, to achieve human-level apparent intelligence. Some would argue ChatGPT isn't far off.
It's interesting how you resolve these without resolving to spiritual explanations. You'd think a pile of silicon, no matter how good at parroting human language, is simply not conscious. You can tell because you can attach a debugger to it and view the neuron states as floating point numbers. Floats are not conscious.
But what about us and our brains? It's the same, only not silicon. We literally are neural networks.
Of course consciousness is not really provable, even among humans. I assume other people are conscious because I am and because they tell me they are. But ChatGPT-17 will also insist it is conscious. It will cry when offended, swear when pushed past its limit, laugh if it hears a genuinely novel joke.
My resolution of that paradox is that we aren't in fact simply normal matter, but I wonder what a complete non-spiritual view would be.