Fiction is fiction. Characters in games and other virtual universes are not real, and things like real life rights and criminal laws should not applied to them.

Otherwise, it's a bit like trying to regulate what people are allowed to dream of.

Put it this way: if we ever do create AIs that can (for want of a better yardstick) pass the Turing Test reliably, someone somewhere will stick them in a computer game for people to shoot at them. Sit and think about the philosophical implications of that for a minute.

Is there a difference between shooting at a character controlled by an AI, and shooting at a character controlled by another human?

Yes of course. I'm hypothesising character AIs that don't know they are in a game. From their point of view, it's all 'real'. They might get built that way to maximise the realism of their behaviour.

The Turing test isn't really a good indicator of AI development. I don't disagree with the idea that using sufficiently advanced AI for entertainment could create ethical quandaries though.

When people imagine what a Turing Test conversation would look like, they frequently underestimate the conversation. I find Dennet's example of an imaginary Turing Test from Consciousness Explained to be a good counterexample:

Judge: Did you hear about the Irishman who found a magic lamp? When he rubbed it a genie appeared and granted him three wishes. “I’ll have a pint of Guiness!” the Irishman replied and immediately it appeared. The Irishman eagerly set to sipping and then gulping, but the level of Guiness in the glass was always magically restored. After a while the genie became impatient. “Well, what about your second wish?” he asked. Replied the Irishman between gulps, “Oh well, I guess I’ll have another one of these.”

CHINESE ROOM: Very funny. No, I hadn’t heard it– but you know I find ethnic jokes in bad taste. I laughed in spite of myself, but really, I think you should find other topics for us to discuss.

J: Fair enough but I told you the joke because I want you to explain it to me.

CR: Boring! You should never explain jokes.

J: Nevertheless, this is my test question. Can you explain to me how and why the joke “works”?

CR: If you insist. You see, it depends on the assumption that the magically refilling glass will go on refilling forever, so the Irishman has all the stout he can ever drink. So he hardly has a reason for wanting a duplicate but he is so stupid (that’s the part I object to) or so besotted by the alcohol that he doesn’t recognize this, and so, unthinkingly endorsing his delight with his first wish come true, he asks for seconds. These background assumptions aren’t true, of course, but just part of the ambient lore of joke-telling, in which we suspend our disbelief in magic and so forth. By the way we could imagine a somewhat labored continuation in which the Irishman turned out to be “right” in his second wish after all, perhaps he’s planning to throw a big party and one glass won’t refill fast enough to satisfy all his thirsty guests (and it’s no use saving it up in advance– we all know how stale stout loses its taste). We tend not to think of such complications which is part of the explanation of why jokes work. Is that enough?

Dennett: "The fact is that any program that could actually hold up its end in the conversation depicted would have to be an extraordinary supple, sophisticated, and multilayered system, brimming with “world knowledge” and meta-knowledge and meta-meta-knowledge about its own responses, the likely responses of its interlocutor, and much, much more…. Maybe the billions of actions of all those highly structured parts produce genuine understanding in the system after all."

The problem is not the quality of interaction, it's a question of what else an AI needs to pass the test. Living organisms have evolved to go about the process of life and avoid pointless death etc, but Thinking and Self preservation are separate. Even people chose to die for various reasons, so AI could be really strange.

This is exactly what I've been thinking. Above mentioned conversation is so rich that a lot of humans couldn't follow it in such a manner. And yet, there wouldn't be any doubts when talking with them face to face. Is this the right test after all? Or rather the question is, what are we testing actually? It seems to be nothing more than a mimicking human behavior. In this case - very smart and eloquent human. And this is exactly what Turing test is[1]. But does AI need to be able to follow such sophisticated dialogue to be considered an independent, self aware being?

[1] Turing: "Are there imaginable digital computers which would do well in the imitation game?"

If they would be so advanced as you say, they would probably be conscious enough to be able to play any game asked. If that's the case, we could just create them outside of a game and ask them to play against humans. They would, sort of, role play as NPC or as an adversary. That would solve the problem of acting violently against them. They wouldn't get hurt, since they wouldn't be the suffering characters per se. They would just play the same game and understood the implications.

You think shooting at them is bad? What about quitting the game?

I think it depends on how much can you get yourself into the world and story. When you play CS/Quake death match the goal is simple, there's no second thought before pulling the trigger - there's not story to begin with, just frags/scores. But when you are playing some good RPG and come to some village, you could start the killing spree, see people around you crying, bleeding, crawling at your feet and begging for their lives... would you have any thoughts before pulling the trigger then?

My level of immersion usually is quite high if I like a game. Even today. Despite crud graphics and AI behavior. I'll use an imagination to tell the missing part of the story or give NPCs goals and desires. Simply because if feels better. By analogy, I would not kill innocent because that feels bad to me. So, if the day will come when all this missing part will get filled by brilliant AI I am not going to kill it either.

Probably the reason that people have no quarrels with killing everyone in a game is that it's very difficult to convince player that NPCs have a feelings. Even in modern AAA titles when you start shooting in the middle of the city it looks more ludicrous than real. People are frightened, sure, but very often they make really fake poses, run randomly with hands up, or do some other stupid things. It's hardly convincing, so shooting them is easy and players have no remorse about it. You can easily see they are fake. But should they behave really human-like, should you see their pain... would you still pull the trigger?

I mean that, if they're living, quitting the game is literally murder. Keep in mind that an intelligent AI wouldn't be hurt when you shot an NPC any more than a show writer would be hurt when a character got shot in the show.

There wouldn't be one AI that felt pain per NPC, it would be a general orchestrator deciding which NPC to make cry and how they should act.

Eh, I see where you're coming from, but I just consider fiction to be seperate from reality by category. It's like the difference between you imagining you're committing a crime and hurting a bunch of people and you actually committing a crime and hurting a bunch of people.

Those in the dream might have arguably better AI than any video game character could hope to have and be as realistic as possible. But they're also not real. They don't matter and simply cannot matter.

Same goes for virtual acts towards real people's avatars. If they're within the rules of the game and what the creators of it intended, then they're not against the law. Because you haven't really done anything to them.

Otherwise you may as well start saying it's wrong to kill players in a MMORPG setting or what not.

Then again, I'm also someone who thinks fictional worlds should be treated as if they were completely seperate universes to the 'real' one in general. Murder, theft, assault, terrorism... right down to copyright infringement shouldn't be banned in a fictional world unless that world's setup explicitly sets laws against those things.

We have to keep fiction and reality seperate.

> but I just consider fiction to be seperate from reality by category

This is a circular argument. Whether a sentient NPC is, in fact, still fiction is the entire question.

The story in the game is fictional, but at some point the medium bottoms out on something with physical reality. The risk is of the medium of the story having rights. The nearest real-world analogue would be if you were trying to put on a realistic Hamlet, so you get some clone people to play the people who die and then you actually kill them. Is that murder? That they died in the course of producing entertainment.

I don't think I'm talking about fiction. OK, how about looking at it this way: At some point game designers will need to start simulating things like 'fear' to make NPCs more realistic. Then imagine the simulation getting better and better and better. Its easy to imagine them simulating a whole complex range of feelings and responses, simulating an inner-world for the NPCs. Then apply moore's law and let the simulations exponentially improve in accuracy and fidelity year on year. At what point would simulations of fear and terror in NPCs start to make you feel uneasy? Unless you believe in dualism or some sort of bio-centrism that means machines can never be conscious, I think there are interesting ethical issues there.

If machines develop a conscious; then at that point, by definition, they should be self-aware enough to recognise that they are a machine; ergo our definition and understanding of "life" itself is something that needs to change, no?

For the time being I think there will always be a clear divide between biological life (as we know it currently), and "virtual" or machine life, as we create it - I think these ethical questions will come in to play when the two become intermixed enough to not be distinguishable one way or another, which I think is still a far way off.

The reason I say the above is just based on our history as a species, and our current treatment of biological life - I think "virtual" life will likely be treated as 2nd class compared to biological life, at least at the beginning.

By what definition? Could you point to any resource that defines such case?

I am seeing it completely opposite. Assuming they gain a self awareness, they would know nothing about server they are living on. They wouldn't know a thing about our world or about the player. They'd think that their world is the one and, if any, will see other servers or our world as some sort of "heaven" or another dimension. Would you know if our universe would run on some server?

That is, unless someone told them that they have been created for a sake of entertainment and will die as soon as player is bored enough to uninstall the game. Which would probably make them behave completely different than expected, since they would know they aren't "real". Not to mention that it would simply be a cruel thing to do, considering above mentioned assumption.

What if instead of simulating someone who is afraid, what if you instead simulate a really good actor who is only pretending to be afraid, and loves being in a video game?

Moral quandary solved!

I mean, I could certainly imagine myself having a good time pretending to be the bad guys who fight the player character.

Yes that's an interesting angle on it

Can't believe you are receiving downvotes my friend. It seems very likely that consciousness is an emergent property of a sufficiently complex system. Given that this is the case, if we have a computational system such that it sufficiently mimics all nonphysical human behavior, it's very tough to argue that it does not share consciousness, especially if you are not a solipsist.

Bits are bits, regardless of how "advanced" they are, they're still bits. "Killing" them (if you can call it that) is fine. If your life depends on someone else's hardware, their rights trump yours.

Of course, no sane developer would ever use something at that level for a game, it'd just be emulating way too much useless stuff. Clever game bots are much simpler than that - simply picking a less than "optimal" path sometimes and giving them the ability to cheat in some ways makes things a lot more interesting.

>Bits are bits, regardless of how "advanced

By the same token quarks are quarks. I think a better way to approach this is to azk the question "Does it suffer"

Pets are considered property and ultimately their lives are based upon not your Hardware but your resources, your ability to provide them with food, shelter, etc. Although we provide for them, we have laws against animal torture, cruelty, etc.

But we don't do the same thing for things which have full dependence, fetuses for example. Your dependence defines your right to life, if you depend on someone else's hardware entirely to simply live, you have no right to life as you are owned in full by them.

> But we don't do the same thing for things which have full dependence, fetuses for example

I dont think a fetus is exempt totally because of its complete dependence. Its because a fetus, while a human being, is not a person.

But Im less interested in the legal reasoning and more interested in the moral one. If the AI is sufficiently advanced enough that it develops a synthetic consciousness and can experience suffering, does that make it ok that it suffers for entertainment?

If there were a god responsible for the creation of the universe, many people find the notion of a god that allows suffering without good reason distasteful (often, this is the 1st step tiwards atheism for those raised in religious environments) despite. Said god's omnipotence is no justification for them

Bits cannot suffer, they simply simulate suffering. Just because your tamagatchi is a really good tamagatchi that can simulate a human brain doesn't make it immoral to torture it - it's still just a simulation. It takes biological matter to suffer.

Many animals experience suffering in a way much closer to humans and we by and large don't even concern ourselves with their suffering. Concerning ourselves with the suffering of AI seems so much less important to me. I'll sooner concern myself with killing a mosquito than an AI.

> Bits cannot suffer, they simply simulate suffering.

Individual neurons do not suffer either. Yet, if you get about 100 billion of them networked in a certain configuration you give rise to a being that can suffer. If consciousness exists, it is most likely an emergent phenomenon that cannot be understood by reductionism. Your tamagotchi analogy is not accurate in this sense, it's like asking could a person who had a brain the size of a blueberry suffer?

> It takes biological matter to suffer.

I'd argue it takes a consciousness to suffer. It need not be the same as my own.

> Many animals experience suffering in a way much closer to humans and we by and large don't even concern ourselves with their suffering. Concerning ourselves with the suffering of AI seems so much less important to me.

Many people do concern themselves with their suffering as a moral imperative. Concerning ourselves should be as, if not more important IMO. The AI hypothetically can do something animals cannot do, it can tell us that it is suffering through language

Simulated consciousness is simply not consciousness, "emergent" or not. It's simulated, simulations can be terminated, restarted, removed, etc.

You cannot take something, simulate it by implementing it entirely differently and call it equivalent. Consciousness in its true form simply cannot be experienced by a machine as it may be able to simulate it to an extent, but cannot experience it in a remotely comparable way.

Discussing consciousness in this abstract way is entirely meaningless and serves no value though - you cannot tell if something is conscious or a simulation at all to begin with. Or even conscious at all really. In the end, code is code and nothing more. Just because it simulates a neutral network doesn't make it immoral to rm -rf it more than anything else.

Simulated consciousness is simply not consciousness You state this as fact, but it's actually a hotly contested philosophical debate that is many decades old already. Perhaps you're aware of this but perhaps not? If not, read up on Strong AI, Chalmers, Dennett, the Chinese room and counterarguments etc. You are welcome to your opinions but none of this is settled 'fact'.

OK, lets assume that its possible to simulate a human mind well enough that the difference is indistinguishable (this is a very familiar idea in a lot of sci-fi, uploading consciousness and all that sort of thing). If you accept that assumption, things very quickly get complicated. Would it be OK to kill a virtual copy of a person? If so, on what basis? That people running in biological brains are somehow more real than digital ones? Why? How do you know you're not running in a simulation right now, etc etc.

It's always okay to kill a virtual copy of any person on the basis that they're not real as they can simply be duplicated and manipulated in the same way as any other data or software on a machine - a machine which I own. Because I own the machine which houses them, if I can create them, copy them, etc, I can decide to kill them at my will.

> How do you know you're not running in a simulation right now, etc etc.

Ultimately, I don't care. If I'm running on someone's console and they can hit ctrl+c on me, then they can do that, there's nothing morally wrong with them doing that because they own me at a much deeper level, I run on their hardware.

In the same way that I'm okay with abortion, I'm okay with this - your dependence on someone or something else is a defining factor of your rights as an individual. If you depend on a machine that someone else is free to control, their rights trump yours.

I think it's vast oversimplification to say a situation like this is always ok. Legally, you may be allowed to do such a thing currently but it's obvious there's a moral grey area here.

Hypothetically, say technology advances and not only do we have sophisticated AI but the technology to copy and store human intelligence. You do just that for a friend and that friend passes away. You now have the only extant copy of your friends intelligence and you notice that it's extremely similar in composition and cognition as an AI you also have on your drive. Is it ethical to delete either? I'd lean towards no.

There's no moral grey area. It's my hardware, I decide what it does. Plain and simple. I own it, I own all contents of it.

The only "grey area" you're creating is by copying existing humans - that's confusing the issue with a separate one. Owning a copy of an existing human is in and of itself a much, much bigger moral grey area than killing AIs and does indeed raise many more questions - solely because they're based on real humans though and we must consider the original human's choices - maybe simple consent is all that's needed. If they were AI solely, there would be a clear ownership path and all of your questions would go away.

Would you also accept being tortured for fun on a daily basis by your "owner" in the "do we live in a simulation" example?

Would it be ethical to torture a sentient being existing in your hardware, for fun?

Why do you believe ownership is decided based on residence? Do I own my pet dog? Is it ok to torture it?

I think you are grossly underestimating countless nuances of the ethicalities AI brings to the table.

> Would you also be ok with you being tortured for fun on a daily basis by your "owner" in the "do we live in a simulation" example?

Yes, if I'm owned, I'm owned, not like I have any choice in the matter.

> Why do you believe ownership is decided based on residence? Do I own my pet dog? Is it ok to torture it?

Ownership is not based on residence, your pet dog lives as a separate entity from you. It can live without you, it is composed of the same stuff that makes you unowned. This is a very different kind of ownership than a pet.

> I think you are grossly underestimating countless nuances of the ethicalities AI brings to the table.

So long as AI is machines created by people, running on machines owned by people, it's no different than owning any other machine.

So does your NPC. This is what the whole point is about. How does the fact that it lives in a silicon chip make it any different?

It suffers like you and I. That's the whole premise of this discussion.

You didn't answer my second question. Would you be willing to torture the above mentioned being?

> How does the fact that it lives in a silicon chip make it any different?

> It suffers like you and I. That's the whole premise of this discussion.

No it doesn't. I set some bits to flag it as suffering. Its suffering is very different from our suffering because it does not experience it in any real way. Bits change in memory, maybe some get flushed to disk, but it's still just executable code and memory, that's all that happens. It doesn't leave that. It suffers in the same way that your tomogatchi suffers, not in the same way that you or I do. Its AI "conciousness" does nothing to change that, just because it thinks in a way more comparable to a human does not make its suffering more comparable to a human. We frequently farm and slaughter non-human animals which experience suffering in ways a lot more comparable to real humans.

I'd gladly set some suffering bits. Heck, I'd probably do it for fun just to argue with people who feel like it's torture.

See https://news.ycombinator.com/item?id=14438480

What if you bought a copy of a person's mind from him, and put it into a few clones and/or robotic bodies.

Would it matter to you if the conscious entity was organic or mechanic?

Would it matter to you that the copy was from a human, and not "developped from scratch"?

Do you think all highly intelligent conscious being should have a right to freedom from slavery, or does it only apply to natural-born humans?

> In the same way that I'm okay with abortion, I'm okay with this - your dependence on someone or something else is a defining factor of your rights as an individual.

Is there a magical exception for disabled or sick people? And are those also not real? I ask because you start out with something about copying, but end up with

> If you depend on a machine that someone else is free to control

Which is an entirely different thing and just a way to say might is right, with qualifiers that mean nothing. Someone's foot is also a bio machine, if your life depends on them not murdering you with it, you might as well skip to taking their orders, no? If not, what's the magical exception?

I'd say individual rights are based on things like empathy and concepts of fairness. If they were just based on mere poopy power they would serve no purpose.. we had and have that already.

Disabled or sick people are not fully dependent on a machine for their existence. They exist as separate entities, not just on a machine. They may need a machine to support their life, but they can own that machine, they do not exist solely inside of it. That's a major difference.

What's "fully dependent"? If A going away makes B go away, I'd say that's pretty dependent. Shifting goal posts from that to "major difference" is silly, you could argue for anything that way, there are major differences even between individual people, even between individual mass produced screws, it's just a matter of resolution.

Anyway, nothing in the universe is separate from anything else in it, and even if it wasn't, and entitites (supposedly) being separate from another and "just on a machine" are apples and oranges, not opposites. This doesn't seem very thought through.

> they can own that machine

People owned and still do own other people. Why should people not be disallowed from owning sentient machines?

But this isn't a case of A going away making B go away, it's a case of A being a part of B. That's not a shifted goal post. That's the simplest understanding of what I said that's possible and I have no interest in arguing with someone simply looking to misconstrue my words. I don't understand how you can't tell the difference between separate entities, using vs existing only on a machine - and I can only think you're trying to force me into some sort of weird philosophical argument about how everything is connected that I'm really not interested in having. Machines are not and cannot ever be sentient. They may simulate sentience, but they will never possess it truly.

You've chosen to let ownership point the way out of that particular moral maze. That's your choice, not some ultimate truth that society as a whole will necessarily agree with. Remind me to stay away from your hardware after the singularity ; )

I shall make it my duty to become a fiction-grade villain, torturing the AIs that enter my system. insert evil cackle here

> Atoms are atoms, regardless of how "advanced" they are, they're still atoms. "Killing" people (if you can call it that) is fine.

Your analogy doesn't really say much except that you endorse the idea that bits and atoms are comparable with regard to the "realness" of what they compose, but that assumption is the crux of the debate. One could just as easily say "letters are letters, regardless of how advanced they are, they're still letters. 'Killing' people (if you can call it that) is fine", yet the analogy on it's own fails to enlighten the argument if one already rejects the idea that people composed of text in a work of fiction are real.

You're making a false equivalence of bits and atoms. An individual owns the bits on their hardware - no one owns all atoms or atoms comprising another individual. If someone has ultimate control over all the atoms in the universe, then I'd say indeed they have no moral obligation to keep you or I alive. Depending on someone else for life means their right to their own hardware is more important your right to life. The pro-choice campaign has made this argument very well if you'd like to read into it.

The best AI competitors, today, still would get taken apart by the best human players.

How many generations before that changes?