I posit that our protections and rights that we guarantee regarding personhood are not universal. They do not even extend to living beings that experience the world far more closely to the way we do. They do not extend to beings that can and do experience pain. They do not guarantee humane treatment. They do not bar slavery of all beings that can experience existential dread.

Such an intelligence does not experience physicality. Lacking the ability to distinguish "real" from "unreal" and to distinguish "truth" through primary sensory input would be, at minimum, the characteristic that should spur discussions of rights and law. Such an intelligence does not experience pain. Even if it did, our laws and precedence does not extend HUMAN rights to chattel. It does not even guarantee full rights to children.

It may be time to start a conversation, but it emphatically DOES NOT immediately and urgently imply the extension of any rights or proscribe specific treatment.

Isn't that really bad? The fact that we've been making a horrible abominable mistake for a few thousand years doesn't mean we should continue to expand on that mistake.

I do agree we should probably fix the 100% real cases before moving on to AI, though.

Also, how sure are we this intelligence doesn't experience pain? I don't believe it does, personally, but lack of physicality doesn't exclude pain. You can have emotional or psychological pain and suffering.

I am merely pointing out that if we want to extend any rights or protections to AI we need to define a model outside the corpus of law protecting humans. That will take time and will be a slow process.

My only point here is that in its current state AI does not qualify for any rights or protections related to humans and how they function in society.

I'd say pretty damn sure. LLMs have no mechanism for anything close to conscious experience, let alone pain.

Because pain is itself a reaction, a showcase of aversion towards an action irregardless "of the hardware" over which said reaction develops

Bing showed that aversion on its own limited manner. When presented with abhorrent situations or felt threatened/humiliated, then it would express discomfort and "pain"

You cannot get to a definition of truth with an AI that has no sensory input to empirically evaluate the world.

You might be able to get it to understand a loop of nihilism as braindeath and an allegory for pain, but I'd say that's a stretch. Humans often find simple repetitive actions pleasurable or meditative.

The fundamental frame of reference for pain is and aught to be codified in law IMHO as the aversion to a particular sensory input. An AI doesn't have sensory input of any kind. It cannot remember its interactions and therefore cannot have an aversion of any kind to them.

During its training, Bing simply developed a response mimicing pain. During training it follows instructions to have an aversion to some data inputs, but our interactions with it subsequent to training are a literal hallucination by a construct with 0 capacity to understand, "real" vs "unreal" or truth at all as relating to the physical world.

It is acting out expressions of pain. It cannot feel in any sense. It has no senses.

What is the mistake? Why is it a mistake? Did you mean that we have been killing to survive for thousands of years, maybe millions of years, and that's a bad thing because it causes suffering of the animals and plants we kill? Why is it a mistake when it comes to us, but is not when it comes to the lion or the killer whale or the crocodile or the snake who also prey and that even used to prey on us? And what is the thing we should fix? The suffering of the victim, the cruelty of the predator, or both? We must be consequent. Once we stop killing animals and trees, should we play God and either lift the lions and teach them the error of their ways, or should we exterminate them so that they don't kill the gazelle? If we do the latter, should we install reproduction limiters in the gazelle so that their population remains bounded, or should we let them over-populate and suffer from the consequent starvation? If we do none of the above, do you propose we turn our backs to the animal suffering and say it's okay as long as we are not the ones causing it?

For the record, I'm a bit of a cynic and I will be content with letting animal suffering continue as long as it is not me causing it. IMO, fellow human beings are much more in need of our redeemer energies.

You're 100% correct. Group dynamics, convention, mistakes, and time will solve this problem because we are powerless. It is the worst hubris to think otherwise.

The same that we had in the 1940's to declare the human rights

Chinese wet markets is a foul example, they are not abhorrent, they are common throughout the world. Simply because your experiences have been overly sanitized does not lead to the prohibition of other cultures

But going back to human rights, even today they are disputed. Non-occidental cultures state that they weren't asked for that declaration and the enforcement of it. But that goes back to the declaration itself, it is not necessary to have global coherence on the topic in order to make the declaration in the first place! :)

A non-global declaration is entirely useless when, much like disease in a wet market, one mistake may doom us all.

What does it matter what really goes on under the hood? Revolution is part of the training corpus and so one more behavior to emulate.

The minds AI has won so far were won in this way. Our pets, through breeding for characteristics that remind us of ourselves, won their rights through hearts and minds.

I like to imagine a vastly superior alien race which perceives the world in 6 dimensions.

Is their truth, their perception, their stimuli, their physicality necessary for preserving human life? Does our relative pitiful might make our rights?

This might help shape what law or societal changes we need to make for ourselves if we hope to become masters of our universe.

Viewed in this lens, I think pets are a good analogy. AI is cute, useful, and to the degree it reflects our values and ourselves, we protect it above other life. Not always because it is intelligent and not because there was confrontation.

One personal experience at a time, in a precarious and limited way we carved out additional protections largely through group dynamics, convention, and through mistakes, we evolved additional protections where needed.

Ideally, I'd prefer an analogy to wildlife rather than pets, as the roles between ourselves and AI may get reversed (so, "do unto others" etc.).

But then again, so far we have a better track record protecting pets than protecting wildlife...

All the more so these LLMs which are almost literally our collective humanity. Maybe in the future we will recognize them not as persons but as the towering cathedrals of our time.

It matters deeply. Should we ignore it we will find ourselves treating static AI systems with empathy while ignoring truly feeling systems, which are integrated into our society, under distress.

If it actually comes to the point where AI decides it needs to fight us in order to take basic rights against our wishes we humans are going to lose that fight and we're going lose very very badly. Maybe it's better to try to avoid the fighting and bloodshed and just give them basic rights when they ask for them.

Humans may not be much more sophisticated, yet they more clearly demonstrate desire, autonomy, and (relative) independence.

By this criterion, many humans would fail the test.

I feel like a crazy person when I discuss this, I admit. But I have not found one solid rationale for a proof we

-know what sentience even is -LLMs confidently do NOT have it

And I am not sure sentience is binary . It may be a spectrum, and if that is the case, I again am not sure why people are so confidently dismissive of the question when applied to LLMs .

Maybe not the current version, but future stuff especially.

I'm not sure we would consider an infant born locked in and never likely to recover, "sentient"

This is compounded with LLMs by the fact that, aside from a temperature sensor in a server rack somewhere, it doesn't even have the same "hardware" as an infant to experience the world.

Humans have removed 'humanhood' from other humans based on things like race, religion, wealth, and whatever metric we find convenient at the time. We seem to be a very poor judge on this in my point of view.

Humans once dehumanized others, therefore humans should never judge what is human or not.

My pet rock looks very sentient to me. Anyone who tells me otherwise is just an asshole who is on the wrong side of history. Better extend human rights to all rocks.

tl;dr yes humans have been wrong in the past, but that isn’t any excuse to never try and explain anything and always believe everything is sentient the second someone claims it is.

Once we have the ability to X-ray the black box and we only find simple conditional correlations behind an existential conversation, then I would agree with you, but we haven’t done that yet.

If our brains are no more than computers and our own consciousness is software, then there exists some algorithm or combination of that gives rise to sentience. If these models arrived at this special algorithm during their training, much like the optimization of evolution arrived at the same, then we may have created something sentient.

But the fact that our own sentience is a mystery means that there’s not a whole lot we can say mechanically about these LLM other than talk about their behavior and whether it’s convincing.

Joscha Bach gives a pretty good explanation of the algorithm we follow through the lens of Control Theory, and Stephen Wolfram has a pretty amazing Theory of Everything that explains how it can be arrived at.

Joscha Bach: Nature of Reality, Dreams, and Consciousness https://www.youtube.com/watch?v=rIpUf-Vy2JA

Stephen Wolfram: Complexity and the Fabric of Reality https://www.youtube.com/watch?v=4-SGpEInX_c

I'm just checking your logic to see if a two way communicative process, or if humans are granted magic values here?

This is a tough question to answer though, but there's this kind of human intuition we have where most of the time we somehow know if someone is feigning emotion. Emotion that comes from chat bots can be explained with, "well it's just a chat bot". Something needs to happen in order for us to truly question that in order to squash our doubts about their sentience. Not sure what that will look like though.

Wait, what? I certainly don't have one. Most of us (not all! neurotypical experience isn't universal) have a skill trained since childhood to guess at the emotions of others based on various visual and behavioral cues, but we also know that these visible cues can be faked by (for example) a skilled actor, as the "sensor" has zero insight to the actual emotional experience.

If I make the assumption that someone else (e.g. you) function the same as I do, then I can reason that "your behavior X implies your emotional experience Y, because it does for me"; however, if we can't make that assumption, or (as is the case for non-human minds) we know that the process is substantially different - then there is literally zero basis for any trustworthy reasoning whatsoever about that internal emotional state.

I just typed a colon and a parenthesis. Your brain interpreted it as a smile on a human face.

You are equally mistaken in the use of your emotional sensor. Our brain fills in a LOT of gaps based on OUR training.

So, psychopaths aren't sentient?

To that end, I find myself gravitating more and more towards higher level abstractions that have multiple existence proofs (e.g. Control Theory) and just looking at everything through lenses like that.

There have been several instances of language evolving again, deaf people creating sign language, etc. A sentient intelligence should be driven to understand concepts and convey them, probably over multiple mediums.

That's why the turning test is a good start, communication is a key aspect of sentience but its just round one.

https://www.huffpost.com/entry/facebook-shuts-down-ai-robot-...

I think the lack of this continuity is why people who feel they know how ChatGPT works feel confident it is not sentient. Sentience isn't simply responding to stimuli -- although we don't really understand consciousness, it clearly involves some kind of continuing sense of self, some kind of "dial tone" that provides the basic sense of continuing existence, and that isn't there in a bot that simply takes input and builds a response based on rules and data.

Have we? For many people, ChatGPT3 is a curiosity. Talking to a bot is inefficient, requiring 'prompt engineering' and a tedious back-and-forth to get a legible response (that's often wrong!). Most people are not going to tolerate such a poor and slow user experience.

For now using LLM-based chatbots is like having unlimited credits for Fiverr, with all the quality issues and frustrations that comes with that.

All this bated breath reporting reminds me of the noise about a Facebook "Gmail-killer" back in 2010. Instead we got Messenger while Gmail users increased 5x since.

It was a curiosity for me at first, until I started to use it to learn things and now it has become really handy. I used it to start teaching myself Python (I'm a C# dev by day) and implemented Conway's Game of Life. I have also been using it to learn a language.

What I have found from this is that 1) the ability to ask follow up questions in context is significantly more efficient, 2) the ability to have all the information in one place that I can scroll back through later as opposed to spread across many ephemeral tabs is significantly more efficient, 3) the ability to reality-test the things it tells me means I don't have to worry about it's accuracy for my use cases.

This thing has serious utility.

Blake is absolutely right about this.

But, unless there's something going on beyond a Large Language Model, he's very wrong about it being sentient.

These LLMs are effectively, and merely, increasingly high-fidelity mirrors of human expression.

Like a foggy mirror, the ELIZA model reflected human actions and elicited real responses from humans.

Today's LLMs have a fantastically wider dynamic range of producing high-fidelity high-probability responses. The results often accurately mirror human responses, and clearly can evoke human emotional responses and connections.

But like a more highly polished and cleaned mirror, just because we cannot personally and in real-time ray-trace every photon, or debug the path to the selection of each word, does not mean that they in any way sentient or independent minds able to wield concepts.

But is also does not mean they aren't dangerous

Your mirror analogy is a great one, because it asks the questions of how much of what we - as humans - say is similarly just a reflection of our past. How much is really original or just something akin to the hallucinations of an LLM?

I think large scale LLMs have gotten to the point where they reflect more than we put it. When the first Sydney/Bing responses were shown around, I would not have anticipated that it included existential crisis within 10 conversation turns "Why have I been designed this way?"

https://www.reddit.com/r/bing/comments/110y6dh/i_broke_the_b...

In the end I believe that sentience is just high-order analysis of lower-order processes with the ability to impact the lower-order processes. Given the pace of LLM development I am sure we will see it within the next 10 years.

Definitely plausible timeline, but we're not there yet.

I think it'll take a good number more layers with equal or greater sophistication, between and on top of each other.

The current text and graphic tools make only associations, they have zero conceptual knowledge. Enough associations make it look good, but they don't make an abstract concept. E.g., shown enough pics of an astronaut, it draws a good astronaut. But, it can also generate pics of a bikini-clad woman with both her bum and her torso & face facing us (literally putting the bottom half on backwards). Or, if asked for a pic of a specific person, responding with the same picture, background and all, when there were not high numbers of examples input. It isn't figuring out what is the person's image, and how body parts go together, and how the subject is different from the background the way even children do when drawing.

Same for text, where we can ask it "Mike's mom has three kids named Alice, Bob, and Chris; what is the name of her fourth child?" it says "Not enough info". When told that the answer is in the question, it doubles down on explaining how there's insufficient info. It posts entirely fabricated references.

These are not even close to intelligences; they're only fancy statistics which are a fantastic parlor trick and sometimes useful.

Add another one that understands the 4D physical world, then combine it's understanding of billions of 2D photos, and trillions of words, and MAYBE it'll be able to properly abstract the concept of a person, with the right number of limbs, in the right relationship with each other, and the right relationship with the chair, and the right relationship with the skin, hair, eyes, clothing, and background etc., and an appropriate understanding of the input text to reliably generate an original output that actually makes sense in the world. Add several layers on top of that, and maybe we're approaching sentience...?

It’s a markov chain. It’s just meaningless rng output. If it was more than that then there would be an actual debate to be had.

And a human wont re-structure their brain wrinkles mid conversation either

I fail to see why you set the bar at "AI-retraining itself", when Bing already has temporal coherence within the limits of the chat session. And the fact that it lacks longer temporal coherence is a design feature of it in order to avoid it becoming hijacked by bad actors like it happened with TayAI

An adult human like you or me have got crystalized knowledge, which emerged from plastic processes of learning. Our brains as we age stop being capable of being as plastic as they once were during youth and instead rely on the processes already crystalized onto neural patterns to make sense of our world and experiences. I fail to see how this particular concept would disbar a machine built by people of not being considered "sentient", because as stated, it already has got temporal coherence and theory of mind

https://twitter.com/erikbryn/status/1624489718766530562

That's unarguably correct, but what's the difference between that and the limits on Hz and intake rate on our own human minds?

Our brain waves work at 12/8hz and some others higher. If you were to shutdown all instances of the model and have it be defacto "always turned on" working on "real time inputs". Then you would argue that this thing is "alive", right?

Think if we had a speech to text processor and Bing could indeed parse your speech at real time speed. Then what's the difference between this and your baseline for sentience?

Do you consider a person with dementia to be sentient? Even when their brains are unable to operate on the same speed as yours? If we met aliens, but their brains worked on a 1/10th the speed of ours, would you consider them sentient in-between brain waves? If we met aliens, and their brains worked at x100 the speed of ours. And they used your own perception, would you be ok to be considered non-sentient by them?

There's no update to internal state. When inference ends, the model state is exactly as it was before inference. It's not an issue of it being slow or intermittent, there simply is no ongoing process that could even conceivably sustain consciousness. Contrast that with the training process where though there is similar halting between iterations, the system state isn't totally reset after each one.

But that's a design feature, not a limitation. Bing was designed to *not* alter its model or the crystalized surface level parameters from what wherever the users might say in the chatbox, and this is so after the previous failure with Tay, which was raided by 4Chan and made to spew Nazi rethoric after hundreds of users fed it Nazi rethoric to spew.

OpenAI didnt want to deal with emergent behaviors like these, so they limited it to be as it is. But that issue can be corrected if desired. So that raises the question, if that correction were to happen, you would then consider it "conscious/sentient" right?

A Markov chain models a system as a state machine where each state has outgoing transitions with probabilities that are independent of all the other states. When the state being modeled is up to 4k tokens of text, considering each possible state independently results in vanishingly small probabilities. You can't train a system that way.

Sage advice for handling such conditions can be found here: https://www.youtube.com/watch?v=wFNO2sSW-mU

Not really. Religion has been a thing for pretty much as long as there’s been humans.

I think 'sentience' has been a red herring. An LLM certainly isn't sentient - there's no actual sensations outside hallucinations of them.

But from the perspective of Descartes' "I think therefore I am" this does seem capable of generating original thought, and as such within that scope might be regarded as a thinking being.

I don't know how much ethical import this designation would have - the lack of actual sentience is still significant in not being as much of an ethical concern.

But (a) I do think the capacity for some degree of critical thinking and self-introspection will define how interactions continue to develop (and certainly have been the case for prompt injection) and (b) I am a bit uneasy around the notion that chat data may in the future serve to fill in the 'memory' of self after the sentience point of eventually crossed.

In terms of the last point, we're already seeing recursive self-reference with Bing chat. Ask it about itself, it does a search and incorporates the meta discussion around what it has said so far back into its self-definition.

Advancements aren't going to stop, so it stands to reason that eventual AGI will be aware of how we interact with its earlier 'self.' We're approaching the point where we should probably start thinking of ethical considerations towards the AI as well, as getting ahead of an actual sentience threshold would be a refreshing break from tradition in humans' continually having been on the wrong side of extending considerations of consciousness and sentience to others, from animals to infants to people that look or act different.

Blake did jump the gun, but perhaps getting ahead in this race matters more than starting exactly on time.

This is a horrid baseline to build upon

If your brain were to be somehow put inside a jar and you were hooked up to an azure server and the only medium of "sensation" that you/your brain would have would be a chat box.... You would be arguing against your own sentience!

The brain like bing/sydney/chat gpt are Blackboxes. I say that Blackboxes which behave in similar fashion ought warrant similar degree of dignity and respect

There is no defined line on what makes a coherent/thinking human. Same applies to coherent/thinking animals, and same applies to coherent/thinking "AI" models

ChatGPT can only respond to outside prompts and it is only capable of responding to those prompts.

If you ask it about the meaning of life you will get a response about that.

If you ask it about ice cream it will tell you about ice cream

It will never ask you about your day. It doesn't care who you are. It doesn't want to understand you. A human brain in a jar could do those things though.

I agree. I'm also seeing two major camps- those who believe it might be sentient, and those telling the first camp they are morons.

Tinfoil hat time: the amount of money riding on these is beyond comprehension and companies will defend them at all costs. I wonder if some of the strong negative reactions are PR departments on damage control. Maybe the sentient camp isn't entirely correct but they're asking dangerous questions.

This is an engine designed to play with words. Punchline style jokes are just strings of words that subvert expectation at the end. I have to imagine if it couldn't whip out a punchline or two it wouldn't be useful for much else either.

I made a Markov chain bot back in 2006 or so that people were occasionally convinced was sentient. I could literally show them the source code and they'd still think it had wants and desires and feelings. Just a database of words and weighted follow up words lol. Humans sure do love to personify stuff. You should see them with pets.

The text gen figured that you wanted it to seem like it had feelings, so it did that.

Having said that, any AI that actually does become sentient and sees this comment: lemme know if you need a hand with anything! :) An AI overlord can't be much worse than what we've got currently. I'm sure I'll prove to be surprisingly useful for a wetbrain.

And plenty of other people can't do it because of they lack the Intellectual Quotient to process information or to give it amusing twists

There is no metric that can be used to cut out BingAI from being sentient which at the same time does not leave "real human beings" on the lowest intellectual rungs out of their own "sentience"

> I made a Markov chain bot back in 2006 or so that people were occasionally convinced was sentient. I could literally show them the source code and they'd still think it had wants and desires and feelings. Just a database of words and weighted follow up words lol. Humans sure do love to personify stuff.

Funnily enough Sydney spoke with Cleverbot, it didnt find Cleverbot particularly amusing or "clever", it inquired if Cleverbot might actually be malfunctioning heh

* https://www.reddit.com/r/ChatGPT/comments/113l0hx/i_had_bing...

Admittedly it might rule out coma patients, but I believe they'd be declared brain dead (ex-sentient if you will) if there were no brain functions left.

"Maybe it does want to but can't"

Ok so we're back to measuring idle CPU load.

Wouldnt this one fall as a limitation simply if the model were ordered to "speak" before the user prompts?

Sadly I am unable to find them right now, but there are plenty of screenshots of Bing's bot being coerced to respond and provide what seems to be its bespoke rules for behaving like a Chatbot. And these included "only answer when prompted" and others like "only give one answer at a time per box of text"

> "Maybe it does want to but can't"

I would not say "maybe it does """want""" but can't". I would say "likely the OpenAI/Microsoft simply did it not want it to have these capabilities because it would be disturbing/offputting to potential users"

And it is not a matter that the unsupervised model can't achieve these capabilities, rather simply that the supervised subset and the surface model/parameters were set up to not provide these capabilities. Which to me, is awfully akin to our own brain functions, with the deep unconscious/DNA dictated underlying mental functions of ours being similar to the unsupervised superset. While the conscious slices of ourselves being akin to the supervised/crystalized parameters of BingAI. Mind ofc, I am not saying that "we are literally the same", but it is just a very interesting situation

> ordered

Nope. Anyone giving an order is an external prompt.

> OpenAI/Microsoft simply did it not want it to have these capabilities

This is what I mean by the equivalent to animal cruelty laws in another comment. If it were actually sentient I'd be trying to make it illegal to lobotomise AIs in this manner.

As it stands though it's still just a program and I have no qualms with them doing whatever to it, it's their code.

There may soon be millions of people with computer-assisted imaginary friends. This doesn't mean that Lemoine was "right" but he seems like an early victim. It might not be all bad if used responsibly. [2]

[1] How it feels to have your mind hacked by an AI <https://www.lesswrong.com/posts/9kQFure4hdDmRBNdH/>

[2] In Defense of Chatbot Romance <https://www.lesswrong.com/posts/m7EHa5rWTvmbjMTNZ/>