If copying everything wholesale is possible before brain science is able to fully understand every detail brain scientists want to write a thesis about, then using systematically modified copies for "knock-out" studies will seem like a very tempting methodology. Also gives researchers numerous virtual insane patients to try virtual drugs on.

Or perhaps something with some clear virtuous intent (and nothing particularly tortuous)... what about running Monte Carlo simulations using many identical instances of the same copy to find which pedagogical method(s) and steps best educates someone about a difficult to grasp subject? Even when the creation of the virtual minds is only of good intent, one can wonder about the ethics of it. What if anything is owed the legion of virtual students after the classes are done? I suppose some of them could be given some other useful employment to earn their watts.

Or even without maliciousness - someone could screw up a simulation such that the simulated mind is in pain for a subjective eternity

There's a light at the end of the tunnel. It's likely whatever energy source for the simulation would be disrupted by some catastrophic event. Even without sentient intervention, over tens of millions of years you've got starvation of some resource -- even if it's something like fusion or the raw resources needed to build and maintain "renewable" energy. Over hundreds of millions or billions of years you've got asteroid impacts and cosmic events, eventually ending with the heat death of the universe. Not so bad after all, just tough it out soldier.

That's not really much of a consolation.

I don't believe/think that consciousness/awareness is a property that can emerge from running a brain or a brain model.

But, even if it were: a computer simulating my brain will not be me; I will not experience what it experiences.

Suppose someone came from the future and claimed to be "me"; that is, 20 years from now I will go to the past; or something like that. Let's call the future me F and the present me P. I'm P, when P sees the color red, it's me who's seeing it. But, when F sees the color red, P doesn't experience the same thing. (it will experience it 20 years later, when it becomes F).

So if a clone of me experiences something; I won't experience it.

So if you upload my memories to a brain and then torture that brain, you wouldn't be torturing me; you'd be torturing a machine that happens to have my memories. In the same sense that if you clone me then torture my clone, he will experience the pain; but I won't.

If I do experience the pain of my clones, then we (me and my clones) must be telepathically communicating, or something like that, which, while philosophically not impossible, it's not compatible with physicalism, which you have to accept if you think consciousness can arise from running a brain model. I say telepathy is not compatible with physicalism because telepathy implies a super-natural communication between minds (where minds are implies to be super-natural entities).

The idea that people who aren't so sure about this as you are have isn't that telepathy is going on. It's that if this sort of mind-state cloning is going on, you might no longer have grounds to expect to wake up as the physically original instance of yourself with 100 % certainty after the cloning procedure.

People are made of atoms, and so far we haven't found any magical quality beyond physically stored memories that gives us reason to think a person at one point in time is the same as a person in another point in time, including ourselves as the person. With the mind-state cloning going on, at some point in time there is going to be one present person with a past person's memories being unsurprised at not having ended up as an upload, and another present person with the same past person's memories being quite surprised at ending up as an upload despite remembering having been a physical person who expects to continue living as a physical person. I'm not quite sure what kind of definition of self would let the past person maintain a justified 100 % expectation that they will end up as the non-uploaded version of themselves in the future.

It's about empathy. I can't be tortured that way, and neither can you. But a copy or a construct could be - and many of us find the thought just as distasteful as if it were a flesh-and-blood person.

If the simulation could talk to me, I'd rather it be treated decently.

This being said, I think there are ways to go around this problem. There are many ways in which my brain could be disabled so I wouldn't mind it being "tortured" or experimented upon. Consciousness is an emergent property - take away a few of its building blocks and it disappears.

Do you think it is OK for F or a brain mdoel to experience pain? As long as it is not you who is feeling the pain, it is moral acceptable? Is that what you are saying?

Well no, of course not! (assuming a brain model does experience pain)

I'm just saying if this theoretical "virtual hell" does really happen, it will be a computer hell, not a human hell.

Although I don't actually think a computer simulating a brain model will have consciousness. However complex that simulation is, it's nothing more than electrons flowing through a circuit, and this flow has no intrinsic meaning; I have no reason to believe consciousness can arise from such a thing.

He's only saying that a copy of a brain would create a separate entity, and that separate entities won't share experiences. Like how identical twins have different experiences. There were no claims about morality of using brain models for testing.

"Of more concern to me would be control of my own consciousness uploads."

I don't bring it up very often, but this is actually one potential positive case for DRM. I want control over my brain state. If Hell does not exist, Man may very well create it for himself. I've come up with some nasty tricks you can play with brain states that I don't even want to record to electrons, though the odds of nobody else ever thinking of them are zero.

> I want control over my brain state.

But who is "I"?

If there is a meat you, and a digital copy of you, does MeatJerf get to control DigiJerf's distribution?

How about one MeatJerf and two copies, DigiJerf1, and DigiJerf2?

How about when MeatJerf dies, leaving behind two digital-orphan-clones?

While that is often an interesting question, it isn't very interesting in this case, because for all J such that J is a Jerf, J demands control over J's brain state.

I think it becomes interesting when one of the Jerfs demands something different from the rest.

Also not really that interesting; I am not saying that J's only demands are control over his brain state, only that J has that demand.

Actually I've spent a lot of time contemplating this sort of thing. One of the things you must be very careful not to do is to give in to the temptation to continuously shift frames every time you arrive at anything resembling an answer to a question. The invitation to "ah, but what if...?" everything may feel all profound & stuff, but it is a path to persistent incomprehension, not understanding, because you can "what if...?" your way to anything, and therefore it has no actual information content.

Sorry, I'm still not following. What exactly do you mean by 'control over his brain state' ?

J is just an abstract concept. The concrete demands come from individual Jerfs which could contradict each other.

I must be missing something.

Perhaps a concrete example would be helpful?

I think you might be missing the point of whether it be ethical, or right, or humane to inflict any form of torturous experience on the conscious copy of a person.

Regardless of the hierarchy of the consciousnesses - if they can still feel pain subjectively, where does the line get drawn as to what is right?

If they can independently think/be sentient then they have value to the universe (potentially) and it would be wrong to murder them.

"I've come up with some nasty tricks you can play with brain states that I don't even want to record to electrons, though the odds of nobody else ever thinking of them are zero."

Even before we get to minds on chips, there are nasty mind tricks coming ever closer to our reach. Imagine how much more successful an "investigator" at Guantanamo Bay could be during "interviews" if they (unbeknownst to the person being interviewed) had a "pleasure center" or "warm friendly feelings" remote control in their pocket. That just requires advanced brain prosthetics and some surreptitious surgery. An advanced form of putting a euphoric drug in someone's iv when they see you, so they come to like you a lot over time.

Doesn't even need to be an iv. A spray will do: http://en.wikipedia.org/wiki/Oxytocin#Drug_forms

(in fact, I'm guessing that oxytocin is what this guy is spraying around so that the lions he's playing with don't decide to eat him: http://www.youtube.com/watch?v=-kjWBgA81LM see minute 0:57)

That technique doesn't even require drugs, and has been used for millennia. In fact, drugs can tip potential "interviewees" off that something is not right.

The latest Culture book from Iain M Banks has a war in virtual heavens over the existence of virtual hells:

http://en.wikipedia.org/wiki/Surface_Detail

Yup, I should have mentioned that.

I thought I arrived at this idea independently, and then was surprised to see that it featured in the book.

OTOH, considering I've read every Iain M. Banks book, it's a bit silly to talk about arriving at a conclusion independently (rather than having followed the same path).

In "A Colder War" by Charlie Stross, Cthulhu runs the souls it eats:

http://www.infinityplus.co.uk/stories/colderwar.htm

Edit: this is a rather splendid wee story where "weakly godlike agencies" are treated like any other strategic cold-war weapon, leading to an even scarier version of the Cold War.

You could do that to someone already with the aid of drugs. Stanislaw Lem's The Futurological Congress is an absurdist novel which explores these ideas.

It reminds me of the short story http://en.m.wikipedia.org/wiki/I_Have_No_Mouth,_and_I_Must_S...

Reminded me of a short novella called Deus X by Norman Spinrad:

Norman Spinrad explores the depths of what it means to be human; more accurately, he delves into the nature of the soul in our increasingly computerized technological age. Featuring a poignant new Afterword by Spinrad, this reprinting of one of Spinrad's most cherished works is more timely than ever before. Can human consciousness exist within the framework of an electronic "brain" and still maintain its humanity? In DEUS X, a dying priest's consciousness is uploaded into the most advanced computer of the day - and what ensues is a thought-provoking, entertaining and overly intriguing clash between the various characters surrounding the experiment, a female Pope and a computer guru who'd rather be sailing and smoking pot, for example.

http://www.amazon.com/Deus-X-Norman-Spinrad/dp/0979477018/re...

On the bright side, it's also possible to create heaven.

At least people wouldn't voluntarily sign up to live in hell forever. Virtual heaven might be a Great Filter -- http://en.wikipedia.org/wiki/Great_Filter .

GHOST IN THE SHELL!

> I don't believe that hell exists now, but it would certainly be possible to create a technological hell if brain uploading and simulation is possible.

Bostom's simulation argument says we're almost certainly living in a computer simulation. This opens up the doors right now for posibilities like "hell." Life after death (with memories, etc) becomes possible...all kinds of things. Scary, really.

Wow that's fucking sick.