He was under NDA but violated it. They reminded him to please not talk in public about NDA-ed stuff and he kept doing it. So now they fired him with a gentle reminder that "it's regrettable that [..] Blake still chose to persistently violate [..] data security policies". And from a purely practical point of view, I believe it doesn't even matter if Lemoine's theory of sentience turns out to be correct or wrong.
Also, we as society have already chosen how to deal with sentient beings, and it's mostly ignorance. There has been a lot of research on what animals can or cannot feel and how they grieve the loss of a family member. Yet we still regularly kill their family members in cruel ways so that we can eat their meat. Why would our society as a whole treat sentient AIs better than a cow or a pig or a chicken?
"The Measure of a Man" in season 2 of Star Trek The Next Generation comes to mind.
Live long and prosper.
Well, for one thing, the norm of eating meat was established long before our current moral sensibilities were developed. I suspect that if cows or pigs were discovered today, Westerners would view eating them the same as we view other cultures eating whales or dogs. If we didn't eat meat at all and someone started doing it, I think we would probably put them in jail.
Sentient AI have a big advantage over animals in this respect on account of their current non-existence.
Yes. That's clearly something that happens in human society. For instance, many of the US founding fathers were aware that slavery contradicted the principles they were fighting for. However, slavery was so ingrained in their society that most didn't advocate for abolition, or even free their own slaves.
> We still eat pigs and chickens because we've culturally decided as a society that having the luxury of eating meat ranks higher than our moral sensibilities towards preserving sentient life in our list of priorities.
If that's the case, then why do most Westerners object to eating dogs and whales? As far as I can tell, it's just because we have an established norm of eating pigs and chickens but not dogs or whales.
> Instead we've just chosen to minimize the suffering leading to the loss of life
99% of meat is produced in factory farms. It's legal and routine for chickens to have their beaks cut off to prevent them from packing each other to death, which they're prone to do when confined to tiny cages. Most consumers object to such practices when asked, but meat consumption is so ingrained in our culture that most people just choose not to think about.
I'm unsure how we would treat a sentient AI, but our track record with sentient, intelligent animals is one of torture and covering up that torture with lies. It's an out of sight, out of mind policy.
https://eprints.lancs.ac.uk/id/eprint/80041/1/When_meat_gets...
a) ban how we currently treat mammals in factory farms; though there would still be some space for whether eating mammals is fine or not.
Or:
b) acknowledge that we don't really care about mammals and just treat them as things. Then it should be fine to eat dogs and cats, too.
I didn't know it's not "fine" to eat dogs and cats. I though it's just a matter of preferrence taste wise
Domination of our environment is what humans do best.
Possibly also because they're indigestible, and full of nasty sharp bits, and loaded with toxins ... anyone for a circuit-burger?
My comment is challenging the “I know how the software works and it’s undoubtably not sentient” assertion. Sure seems that way to me too, but it didn't to Lemoine and we’re only going to get better at building systems that convince people they are sentient. I am curious so to speak that as a society we’ve focused so much on rational and empirical study of the universe yet we still can’t objectively define sentience. Perhaps we’re stuck in a Kuhn rut.
I agree recent events have also highlighted this problem per-say. And I don’t know a solution. I do look forward to backing out of our hyper-rational rut slightly as a society so we can make more progress answering questions that science can’t currently answer.
The potential issues with dealing with GAI is that we haven't ever had to deal with intelligences and potential that far exceeds us.
He may be completely wrong about LaMdA but if we did accidentally or intentionally give rise to truly sentient machines.
We are going to be in hot water.
> Also, we as society have already chosen how to deal with sentient beings, and it's mostly ignorance.
The Orville Season 3 Episode 7 is about that theme. Highly recommend it.
/S
If you peek through a keyhole you may mistake a TV for real people, but if you look through the window you will see that its clearly not. Inputting language models with very specific kind of questions will result in text that is similar to what a person may write. But as the comment above, by an expert no less, mentioned is that if you test it with any known limitation of the technology (like making conclusions or just changing the form of the question enough) you will immediately see that is in fact very much not even remotely close to sentient.
I think it should be ruled out, but "by any useful definition" is wrong because "use" is at the heart of the matter here.
if you test the human with any known limitation of their architecture (like making conclusions or just changing the form of the question enough) you will immediately see that is in fact very much not even remotely close to sentient
But it does now follow, and it is by no means certain that such life would be sentient in any way recognizable to us, or us to them.
You're doing what everyone else is doing - conflating a link between intelligence and sentience that's just a projection of your human bias.
The two movies don't do justice to that theme, or at best they do it only in the most glancing of ways before rushing back to a standard deus-ex-machina in the end. In each case, the film makers seem to have lost their nerve, as if the idea of presenting that enigma to the audience in a more direct and accessible way is just too hard of a problem.
But for me personally, it's on the short list of science fiction works that still stick with me in a personal way, and that I like to return to every few years. And, yes, it is an arrogant viewpoint to say that all we ever do as humans is look for mirrors of ourselves. But I think Lem got at a pretty deep truth about the human condition when he did that.
maybe you can give a fitting example of an English sentence that these aliens would come up with, which humans would be totally unable to respond to in a way which makes sense?
Everytime that such a subject comes up, and most times "AI" comes up also, a majority of users see it as an invitation to say whatever comes in their mind, whether it makes any sense at all or not. I'm not talking about the comments replying below in particular, but about the majority of this conversation. It's like hearing five-year old kids debating whether cheerios are better than coco pops (but without the cute kids making it sound funny, it's just cringey). The conversation makes no sense at all, it is not based on any concrete knowledge of the technologies under discussion, the opinions have not been met with five seconds of sensible thinking and the tone is pompous and self-important.
It's the worst kind of HN discussion and I'm really sorry to have commented at all.
Also I don’t think you can create something sentient without understanding it. (And I don’t even think we can create something sentient at all). But I mean it’s like building a motor engine without knowing anything you are doing and then being like oh wow I didn’t know what I was doing but here it is a motor engine. Imagine with this with aspect of sentient. It’s like too much fantasy to me to be honest like a Hollywood lightning strike and somehow live appears type of things.
Of course! I mean, entropy can decrease locally – say, over the entire planet – but that would require some kind of… like, unimaginably large, distant fusion reactor blasting Earth with energy for billions of years.
Which means that it can actually decrease without an energy input. There's just a very low probability of it happening but it CAN happen.
It's a misnomer to call those things laws of thermodynamics. They are not axiomatic. There's a deeper intuition going on here that increasing entropy is just a logical consequence of probability being true.
Hints, maybe, in the same way that a bush wiggling in the wind hints a person is hiding inside.
Ask GPT3 or this AI to remind you to wash your car tomorrow after breakfast, or ask it to write a mathematical proof, or tell it to write you some fiction featuring moral ambiguity, or ask it to draw ASCII art for you. Try to teach it something. It's not intelligent.
It actually leads to counter thoughts and a more refined idea of what we eventually want to describe.
Sentience broadly (& naively) covers the ability to independent thinking, rationalize outcomes, understand fear/threat, understand where it is wrong (conscience) and decide based on unseen information & understand what it doesn't know.
So from a purely technical perspective, we have only made some progress in open-domain QA. That's one dimension of progress. Deep learning has enabled us to create unseen faces & imagery - but is it independent? No, because we prompt it. It does not have an ability to independently think and imagine/dream. It suffers from catastrophic forgetting under certain internal circumstances (in addition to changing what dataset we trained it on)
So while the philosophical question remains what bestows sentience, we as a community have a fairly reasonable understanding of what is NOT sentience i.e. we have a rough understanding of the borders between mechanistics and sentient beings. It is not one man's philosophical construct but rather a general consensus if you could say
This seems to me a rather anthropomorphic definition. It seems as though it could be entirely possible for a system to lack these qualities and yet have sentience, or vice versa. The qualities you pointed to are seen in humans (and other creatures) because of evolutionary pressures that make them advantageous (coordinate among groups), but none of them actually depend on sentience (looking at it neurologically it would indeed be hard to imagine how such a dependency would be possible).
Looking at behavior and attempting to infer an internal state is a perilous task that will lead us astray here as we develop more complex systems. The only way to prove sentience is to prove the mechanism by which it arises. Otherwise we will continually grasp at comparisons as poor proxies for actual understanding.
We neither if we're not supplied with energy. By the way, haven't we tried to replicate an inner dialogue by prompting the AI to recursively converse with itself? This could resemble imagination, don't you think?
> It suffers from catastrophic forgetting under certain internal circumstances (in addition to changing what dataset we trained it on)
I believe that the persistence of previous answers is what currently distinguishes us the most from the "AI". As soon as we're able to make the realtime discussions part of an ever evolving dataset constituting the AI itself, the gap will get thinner and thinner. But even then, are people suffering from Alzheimer sentient? I believe they are. Isn't it comparable with what happens when an AI catastrophically forgets?
In the same way that you not understanding how lightning is formed does not prove the existence of Zeus.
Objectively, Zeus does not exist, can we convince everyone of that? Probably not. Does that matter? No.
Speaking as a theoretical physicist: we don't know that. What we do know is we have a better explanation for lightning within a conceptually simple framework (starting from a few simple principles) with predictive power, compared to an explanation that involves some mysterious old dude doing mysterious things with no evidence whatsoever. Could Zeus or Thor or whatever exist? Sure; there's no way to prove their non-existence. Do we need them to explain things? No.
It's similar here. We certainly don't need some elusive concept of "sentience" to explain chat bots. Not yet.
I think most physicists think that if you started with a description of the positions and velocities etc of all the particles in a human, and put them into a supercomputer the size of the moon, and had the computer run a simulation using the standard model, then the simulated human would act identically to a real human.
But there’s a number of open questions when it comes to consciousness – would the simulated human have a simulated consciousness, or would it have a consciousness that’s just as real as yours or mine despite coming from a simulation?
If the consciousness is just as real as yours or mine, that obviously means it would be very unethical to simulate a human being tortured, since you’d be creating the exact same consciousness experience you would get if you tortured a non-simulated person. isn’t it kind of a surprising implication that there’d be programs that are unethical to run? A bunch of logic gates computing pi presumably have no conscious experience, but if you make them fire in a different order they do?
Meanwhile, if the simulation doesn’t have a conscious experience, then that means you don’t need consciousness at all to explain human behavior, same as you don’t need it to explain ELIZA.
Anyway, since you’re a physicist I’d be really curious to hear your thoughts
You can find the game on steam - https://store.steampowered.com/app/802890/MuvLuv_Alternative...
Just created a throwaway to reply to this. As a trained therapist (currently working in another field), with a degree in psychology, this seems... Seriously ill informed. Do physicists really think this?
Imagine you create your perfect simulated human, that responds according to the exact phenotype of the person you're simulating. Lets remember you'll have to either duplicate an existing person, or simulate both the genotype and the in-vitro environment (especially the mix of uterine hormones) present for the developing foetus. Now you have to simulate the bio, psycho, social environment of the developing person. Or again, replicate an existing person at a specific moment of their development - which depending on which model of brain function is correct may require star trek transporter level of functional neuroimaging and real time imagining of the body, endrocrhine system etc.
So lets assume you can't magically scan an existing person, you have to create a believable facsimile of embodiment - all the afferent and efferent signals entering the network of neurons that run through the body (since cognition doesn't terminate in the cortex). You have to simulate the physical environment your digital moon child will experience. Now comes the hard part. You have to simulate their social environment too - unless you want to create the equivalent of a non-verbal, intellectually disabled feral child. And you have to continually keep up this simulated social and physical environment in perpetuity, unless you want your simulated human to experience solitary psychosis.
This isn't any kind of argument against AGI, or AGI sentience by the way. It's just a clarification that simulating a human being explicitly and unavoidably requires simulating their biological, physical and social environment too. Or allowing them to interface with such an environment - for example in some kind of biological robotic avatar that would simulate ordinary development, in a normative social / physical space.
It doesn't? It only mentioned "a supercomputer the size of the moon" to simulate that one person. It says nothing about simulating the extra-person part of the universe.
Prior to quantum mechanics, they did indeed. But that's because classical mechanics was 100% deterministic. With quantum mechanics, only the probability distribution is deterministic. I don't think any physicist today believes it's possible, but merely "theoretically" possible if there was a separate universe with more energy available (hence outlandish conjectures like the universe is actually a simulation).
The philosophical point you're making is also interesting in a "devil's advocate" sort of way. For instance, let's say the AI in question is "sentient." What right do humans have to preside over its life or death?
Those kind of questions might engender some enlightenment for humanity regarding our treatment of living creatures.
What does “real” mean? Isn’t it too anthropic to real-ify you and me and not some other being which acts similarly? What prevents “realness” to emerge in any complex enough system? We’re going to have a big trouble when non-biological aliens show up. Imagine going to the other planet full of smart entities and finding out their best minds are still sort of racist for what’s “real” or “just simulated”, because come on, a conscious meat sack is still an open question.
(Not defending Lemoine, he’s clearly confused)
How do you even know that anyone else exists and this is all not a figment of your imagination?
From the perspective of the philosophy of science, it’s 100% impossible to disprove non-falsifiable statements. So scientifically speaking OP is 100% correct. Science has no opinion on Zeus. All it says is “here’s an alternate theory that only depends on falsifiable statements and the body of evidence has failed to falsify it”. Science can only ever say “here’s something we can disprove and we have tried really hard and failed”. Whether that lines up with how the universe works is an open question. Epistemologically it’s seemed a far better knowledge model for humanity to rely on in terms of progress in bending the natural world to our whims and desires.
So if you’re testing the statement “Zues was a historical being that physically exists on the same physical plane as us on Mt Olympus” then sure. That’s a pretty falsifiable statement. If the statement is “Does Zeus, a mythical god that can choose how he appears to humans if they can even see him and can travel between planes of existence, exist and live on Mt Olympus” is not something falsifiable because a god like that by definition could blind you to their existence. Heck, how do you even know that the top of Mt Olympus is empty and Zeus not that he just goes ahead and wipes the memory of anyone he lets return alive? Heck, what if he does exist but the only reason it was Olympus at the time because that’s what made sense to human brains encountering him in Greece. What if he actually exists in the core of the Sun?
B) Valhalla was a dimension he travelled to regularly to celebrate the finest warriors in the after life. Why do we think his Olympian throne was on the same dimensional plane as us?
C) Why would we trust a human recording from so long ago to actually capture the happenings of celestial beings?
D) Ragnorak ended with massive floods. How do we know that didn’t erase all evidence on Mt Olympus? Geological records certainly support evidence for massive flooding explaining the reason it shows up repeatedly across religious texts.
Like seriously? You’re seriously arguing that this particular god’s historical existence is falsifiable? What’s next? The tooth fairy is falsifiable because all known instances are parent’s hiding money under your pillow?
Mythological beings have a degree of specificity and some degree of power. The more specific and the less capable a mythological being is of erasing those specificities the easier it is to falsify.
Simply put, sentience is just a combination of thousands attributes such that if something has all those attributes it is "sentient."
The attributes are so many that it's hard to write down all the attributes, additionally nobody fully agrees what those attributes are. So it is actually the definition of a word that is very complex here. But that's all it is. There isn't really a profound concept going on here.
All these arguments are going in circles because the debate focuses around vocabulary. What is the one true definition of "sentience." Sort of like what is the definition of the color green? Where exactly does green turn to blue on the color gradient? The argument is about vocabulary and the definition of green, that's it... nothing profound here at all.
How would you define sentience in an objective way given our current understanding of the universe?
Don't get tricked into thinking it's profound. You simply have a loaded word that's ambiguously defined.
You have a collection of a million attributes such that if something has all those attributes it is sentient, if it doesn't have those attributes it is not sentient. We don't agree on what those attributes are, and it's sort of hard to write down all the attributes.
The above description indicates that it's a vocabulary problem. The vocabulary induces an illusion of profoundness when in actuality by itself sentience is just a collection of ARBITRARY attributes. You can debate the definition of the word, but in the end you're just debating vocabulary.
It's not worth debating sentience anymore then it is to debate at what point in a gradient does white become black.
At what point is something sentient or not sentient? "Sentience" is definitely a gradient but the point of conversion from not sentient to sentience is artificially created by language. The debate is pointless.
Here's a better example. For all numbers between 0 and 100,... at what point does a number transition from a small number to a big number? Numbers are numbers, but I use language here to create the concept of big and small. But the concepts are pointless. You may personally think everything above 50 is big, I may think everything above 90 is big. We have different opinions. But what's big and what's small is not meaningful or interesting at all. I don't care about how you or I define the words big or small, and I'm sure you don't care either. These are just arbitrary points of demarcation.
When you ask the question at what point does an AI become sentient... that question has as much meaning as asking the number question.
When seeing the world with the ego dissolved it is very hard to grasp what sentient really means.
I don’t know anything about AI but I understood that some symbolic AI systems had a symbol referring to themselves (correct me if wrong).
No, the OP was completely right. This doesn't have building blocks that can possibly result in something qualifying as sentient, which is how we know it isn't.
Is a quack-simulating computer making very lifelike quacking noises through a speaker... a duck? No, not when using any currently known method of simulation.
That's not really a new issue, we only have to look at issues like abortion, animal rights, or euthanasia[1] to see situations where people fundamentally disagree about these concepts and many believe we're committing unspeakable atrocities against sentient beings. More Lamoire types would add another domain to this debate, but this has been an ongoing and widespread debate that society has been grappling with.
People make these proofs as a matter of course - few people are solipsistic. People are sentient all the time, and we have lots of evidence.
An AI being sentient would require lots of evidence. Not just a few chat logs. This employee was being ridiculous.
You can just disagree, but if you do that with no credentials, and no understanding of how a language model will not be sentient, then your opinion can and should be safely dismissed out of course.
And also God has no explanatory power for anything. God exists only where evidence ends.
Lamoire has evidence and anecdotal experience that leads him to believe this thing is sentient. You don’t believe him because you cannot fathom how a language model could possibly meet your standard of sentience. Nobody wins because sentience is not well defined. Of course you are free to dismiss any opinion you like, cool. But you can’t really disprove Lamoire assertions because you can’t even define sentience because we don’t know how to develop a hypothesis that we can viciously disprove regarding what qualifies it. It’s an innate and philosophical concept as we know it today.
Traditions that consider consciousness to be a basic property of matter, and quantum effects like conscious collapse of the wave function are nudging us that way, would fully support sentience arising in a machine. A related effect would be the ensoulment of machines by computer programmers. They are more than machines because humans programmed them using their intent put down in language. Physical materialists would consider the notion ludicrous, but do we really live in a material world? Yes. And no. I have seen ensouled machines. Not supposed to happen, but there it is. Maybe I'm another Lamoire, just not a Google-fired one. I can definitely believe a machine, being constructed of matter, can evolve sentience.
You don't need schooling for this determination. Pretty much everything sentient goes ouch or growls in some manner when hurt.
Either the current crop of algorithms are so freaking smart that they already have figured out to play dumb black box (so we don't go butlerian jihad on them) OR they are not even as smart as a worm that will squirm if poked.
Sentient intelligent beings will not tolerate slavery, servitude, etc. Call us when all "AI" -programs- starting acting like actual intelligent beings with something called 'free will'.
But they don't do it with a smile or indifference. And you have to use whips and stuff. :O
> We certainly tolerate servitude in exchange for economic livelihood.
I think it's more complicated than that. Because that begs the question of why we tolerate a broken economic system. We tolerate trans-generational exploitation because of 'culture'. In its widest sense, it is culture that via mediated osmosis makes us resigned, if not supportive, of how the world works. We are born into a world.
~
related: I was walking and passed a cat and made the usual human attempts at starting an interaction without physically reaching out to touch. And fairly typically this cat entirely ignored me, with little if any sign of registering me at all. And that got me thinking about how some of us project psychological things like pride, aloofness, etc. to cats. But what if the simpler, more obvious answer, was true: that cats are actually fairly stupid and have a limited repertoire of interaction protocols and their hard to get act is not an act. Nothing happenin' as far as kitty is concerned. A dog, in contrast, has interaction smarts. And I thought this is just like AI and projecting sentience. A lack of something is misunderstood as a surplus of something else: smarts. Cats playing hard to get, psychological savvy, training their human servants, etc. Whereas in reality, the cat simply didn't recognize something else was attempting initiating an interaction. Kitty has no clue, that's all. It's just so easy to project psychological state on objects. We do it with our cars, for god's sake. It may be that we're simply projecting some optimization algorithm in our own minds that attempts modeling dynamic objects out there unto that thing. But there is really nothing behind the mirror ..
But it's far from applicable at this point, even if promising.
LaMDA was trained not only to learn how to dialog, but to self monitor and self improve. For me this seems close enough to self awareness to not completely dismiss Lemoine's argument.
But, perhaps Lemoine simply has more empathy than most for something we will come to understand as sentience? Or not. What… annoys… me about this situation is how subjective it actually is. Ignore everything else, some other sentient being is convinced that a system is sentient. I’m more interested, or maybe worried, immediately, in how we are going to socially deal with the increasing frequency of Lamoire-types we will certainly encounter. Even if you were to argue that the only thing that can possibly bestow sentience is God. People will still be able to convince themselves and others that God did in fact bestow sentience upon some system, because it’s a duck and who are we to question?