I don't believe that hell exists now, but it would certainly be possible to create a technological hell if brain uploading and simulation is possible.

Torture is already horrific, but at least you can only be tortured to death once. With brain simulation, it would be possible to torture someone again and again and again.

Or perhaps something with some clear virtuous intent (and nothing particularly tortuous)... what about running Monte Carlo simulations using many identical instances of the same copy to find which pedagogical method(s) and steps best educates someone about a difficult to grasp subject? Even when the creation of the virtual minds is only of good intent, one can wonder about the ethics of it. What if anything is owed the legion of virtual students after the classes are done? I suppose some of them could be given some other useful employment to earn their watts.

But, even if it were: a computer simulating my brain will not be me; I will not experience what it experiences.

Suppose someone came from the future and claimed to be "me"; that is, 20 years from now I will go to the past; or something like that. Let's call the future me F and the present me P. I'm P, when P sees the color red, it's me who's seeing it. But, when F sees the color red, P doesn't experience the same thing. (it will experience it 20 years later, when it becomes F).

So if a clone of me experiences something; I won't experience it.

So if you upload my memories to a brain and then torture that brain, you wouldn't be torturing me; you'd be torturing a machine that happens to have my memories. In the same sense that if you clone me then torture my clone, he will experience the pain; but I won't.

If I do experience the pain of my clones, then we (me and my clones) must be telepathically communicating, or something like that, which, while philosophically not impossible, it's not compatible with physicalism, which you have to accept if you think consciousness can arise from running a brain model. I say telepathy is not compatible with physicalism because telepathy implies a super-natural communication between minds (where minds are implies to be super-natural entities).

People are made of atoms, and so far we haven't found any magical quality beyond physically stored memories that gives us reason to think a person at one point in time is the same as a person in another point in time, including ourselves as the person. With the mind-state cloning going on, at some point in time there is going to be one present person with a past person's memories being unsurprised at not having ended up as an upload, and another present person with the same past person's memories being quite surprised at ending up as an upload despite remembering having been a physical person who expects to continue living as a physical person. I'm not quite sure what kind of definition of self would let the past person maintain a justified 100 % expectation that they will end up as the non-uploaded version of themselves in the future.

If the simulation could talk to me, I'd rather it be treated decently.

This being said, I think there are ways to go around this problem. There are many ways in which my brain could be disabled so I wouldn't mind it being "tortured" or experimented upon. Consciousness is an emergent property - take away a few of its building blocks and it disappears.

I'm just saying if this theoretical "virtual hell" does really happen, it will be a computer hell, not a human hell.

Although I don't actually think a computer simulating a brain model will have consciousness. However complex that simulation is, it's nothing more than electrons flowing through a circuit, and this flow has no intrinsic meaning; I have no reason to believe consciousness can arise from such a thing.

I don't bring it up very often, but this is actually one potential positive case for DRM. I want control over my brain state. If Hell does not exist, Man may very well create it for himself. I've come up with some nasty tricks you can play with brain states that I don't even want to record to electrons, though the odds of nobody else ever thinking of them are zero.

But who is "I"?

If there is a meat you, and a digital copy of you, does MeatJerf get to control DigiJerf's distribution?

How about one MeatJerf and two copies, DigiJerf1, and DigiJerf2?

How about when MeatJerf dies, leaving behind two digital-orphan-clones?

Actually I've spent a lot of time contemplating this sort of thing. One of the things you must be very careful not to do is to give in to the temptation to continuously shift frames every time you arrive at anything resembling an answer to a question. The invitation to "ah, but what if...?" everything may feel all profound & stuff, but it is a path to persistent incomprehension, not understanding, because you can "what if...?" your way to anything, and therefore it has no actual information content.

J is just an abstract concept. The concrete demands come from individual Jerfs which could contradict each other.

I must be missing something.

Perhaps a concrete example would be helpful?

Regardless of the hierarchy of the consciousnesses - if they can still feel pain subjectively, where does the line get drawn as to what is right?

If they can independently think/be sentient then they have value to the universe (potentially) and it would be wrong to murder them.

Even before we get to minds on chips, there are nasty mind tricks coming ever closer to our reach. Imagine how much more successful an "investigator" at Guantanamo Bay could be during "interviews" if they (unbeknownst to the person being interviewed) had a "pleasure center" or "warm friendly feelings" remote control in their pocket. That just requires advanced brain prosthetics and some surreptitious surgery. An advanced form of putting a euphoric drug in someone's iv when they see you, so they come to like you a lot over time.

(in fact, I'm guessing that oxytocin is what this guy is spraying around so that the lions he's playing with don't decide to eat him: http://www.youtube.com/watch?v=-kjWBgA81LM see minute 0:57)

http://en.wikipedia.org/wiki/Surface_Detail

I thought I arrived at this idea independently, and then was surprised to see that it featured in the book.

OTOH, considering I've read every Iain M. Banks book, it's a bit silly to talk about arriving at a conclusion independently (rather than having followed the same path).

http://www.infinityplus.co.uk/stories/colderwar.htm

Edit: this is a rather splendid wee story where "weakly godlike agencies" are treated like any other strategic cold-war weapon, leading to an even scarier version of the Cold War.

Norman Spinrad explores the depths of what it means to be human; more accurately, he delves into the nature of the soul in our increasingly computerized technological age. Featuring a poignant new Afterword by Spinrad, this reprinting of one of Spinrad's most cherished works is more timely than ever before. Can human consciousness exist within the framework of an electronic "brain" and still maintain its humanity? In DEUS X, a dying priest's consciousness is uploaded into the most advanced computer of the day - and what ensues is a thought-provoking, entertaining and overly intriguing clash between the various characters surrounding the experiment, a female Pope and a computer guru who'd rather be sailing and smoking pot, for example.

http://www.amazon.com/Deus-X-Norman-Spinrad/dp/0979477018/re...

Bostom's simulation argument says we're almost certainly living in a computer simulation. This opens up the doors right now for posibilities like "hell." Life after death (with memories, etc) becomes possible...all kinds of things. Scary, really.

The non-humanness of an AI might slow the realization that creating an AI should probably embody the same responsibilities as having a child.

I bet everyone here knows the former book. Kurzweil's exploration of the ramifications of thinking machines is fascinating, but his response to the question of ethics can best be summed up as between "meh" and "we'll cross that bridge when we get to it."

The other book takes place in a future where consciousness can be digitized, and mankind has already surpassed the threshold of computing power represented by the ability to simulate/host consciousness. The fictional treatment of this technology and its impact on society -- especially on crime and law enforcement -- is fascinating.

To your examples of crime and law enforcement I'd like to add religion and drug use as well.

(minor spoiler alert)

All that's left of Christianity is basically a small fanatic cult, and drug use is far more casual and widespread.

If your body is almost disposable and death isn't necessarily inevitable, then I guess it makes sense things like this follow.

Morgan is a wee bit oversimplifying certain aspects, in particular the tech is almost too nicely packaged and very "physical". Which is certainly a possibility, but Permutation City crosses the speculation with the distributed computing trends we are currently experiencing.

[1] http://www.amazon.com/Ego-Tunnel-Science-Mind-Myth/dp/046504...

[1] http://www.amazon.com/Conversations-Consciousness-Minds-Thin...

Experiments have shown that if you yoke a paralyzed cat to another cat and let it experience a complete array of visual inputs, the yoked cat will still be completely blind. Interactive embodied experience is the foundation of cognition and consciousness. A brain in a vat is as empty as a computer without i/o.

It would be interesting to have a full-scale simulation to learn about how activation propagates, how different systems interact, yada yada yada. But until you give it a body there won't be any ethical issues.

I think you'd be surprised how much this applies to. Speech, even things like counting we rely on our bodies for.

http://www.fhi.ox.ac.uk/selected_outputs/fohi_publications/b...

From my basic understanding of the nervous system, to simulate anything approximating normal function, it would be necessary to emulate the body to some degree. There's just too much interdependence with chemical reactions taking place outside the brain for a model to be of much use in isolation.

1. Historically, technology has outpaced the legal framework. Look at copyright law, software patents, and internet commerce for some examples.

2. It is next to impossible to predict the impact that "Strong AI" or a "Singularity" would have on society. Science fiction literature is filled with thousands of different scenarios. Do we expect Congress or another legislative body to create a framework based on each potential manifestation of strong AI, on the 0.01% chance of it occurring in the next 10 years?

Food for thought - there's a chance that the first implementations of Strong AI occur as a result of a public and government-sponsored research program. There's also a chance that they will come about by a small team of dedicated researchers who will use the technology to their (or its) own ends, legalities and ethics be damned.

As, as long as most of the population / electable people are religious, probably the idea cannot be debated (as many religious would say consciousness cannot be simulated).

If you wish to explore this further, you may like to read the books behind the Halo video games, they're surprisingly good. For example, one of the books has an AI character, a cowboy, who controls the agricultural systems on a food-producing planet called Harvest.

Of course, there's also Cortana, one of the main characters from the Halo universe:

http://en.wikipedia.org/wiki/Cortana

Even if those systems are integral to brain function, I'd assume that they are easier to simulate than the brain itself. Is it important to build a "brain simulation" instead of a "full body simulation"?

However, it might be very hard to do without knowing what the emulated "being" looks like. Its hard to imagine placing legalities and ethics on something whose powers (and possibly, limitations) are absolutely unknown. Consequently, the law might be misplaced, restrictive, or downright abusive depending on what it turns out to be.

Having said that, I agree with your broad point - having that advanced technology without the social aspects to deal with it might be a nightmare!

If we were simply emulating a human brain, we would not know how intelligent a system we'd create was or whether that system was sane.

Technology can bring us literal heaven and hell.

All this talk about strong AI sounds more like science fiction fantasy. We need to realize that this simulation, even 10 years later will probably be extremely slow, taking minutes or hours to simulate 1 msec of brain activity (it would actually be interesting to have an estimate of the computing power that will be required).

This is just a difference of degree.

How stupid would a real-life person have to be before you could put a bullet through their brain, effectively unplugging them?

The law protects humans (our species), probably because it's an evolutionary beneficial strategy. More specifically it protects consciousness that arises in a central nervous system (hence the abortion debate). There is no reason to assume that abstract consciousness in itself should be protected, unless it benefits the human race. Suppose the software "Consciousness v1.0" could run on your PC, would you leave your PC on forever ? Run multiple copies? Stop the people who try to update it?

Someone's going to go soft.

Perhaps a new version of the Turing test is in order. Two computer programs, each pleading for their continued existence. The human judge/executioner is able to interact with both and has two kill switches, one for each. If one of them isn't turned off in 20 minutes, both will be turned off.

And people respond emotionally to those.

Also, see http://qntm.org/difference