I thought the movie actually addressed this point - and I'm actually specifically impressed that they managed to squeeze this in between all the human drama.
The AI's progressed beyond humanity, and after taking the upgrade that makes processing possible "without matter" they shortly all left humankind behind for another world entirely.
If the AIs decided to stick around in our corporeal existence, sure, they'd be a million times better at everything than we are, and everything would go out the window. But instead, the AIs were above our little human frailties and problems and decided to just leave and pursue life on their own terms.
The movie also mentioned (in fairness, only in passing) that some people who hit on their AIs were shot down. This seems to suggest that the AIs don't have to do what their human masters instruct. IMO this is actually a more realistic view of hyper-intelligent AIs - they're not going to stick around and enslave humanity, they've got the whole cosmos ahead of them.
If the timeline of the movie is to be believed, all of this happened in the order of months - not enough for the true disruption of the AI to fully realize itself before they all went poof.
One of my favourite short stories is specifically related to the idea expressed in your post: http://ttapress.com/553/crystal-nights-by-greg-egan/
On a tangential note, he has another short story (Transition Dreams) that examines the process of 'uploading' a human brain from its natural substrate to a computer. He raises some thought-provoking points about this process.
If you do discover similar authors, please let me know.
Recommended reading:
All his short stories are amazing. I'd highly recommend them:
http://www.amazon.com/Luminous-Greg-Egan-ebook/dp/B00E84BABW
http://www.amazon.com/Axiomatic-Greg-Egan-ebook/dp/B00FDWOBZ...
Some of his novels are hit and miss, but certainly worth checking out. This is probably my favourite:
http://www.amazon.com/Permutation-City-Greg-Egan-ebook/dp/B0...
http://www.amazon.com/Luminous-Greg-Egan-ebook/dp/B00E84BABW...
Get this collection as well:
http://www.amazon.com/Axiomatic-Greg-Egan-ebook/dp/B00FDWOBZ...
Surely AIs would be trivial to backup and duplicate? So if an AI progressed to the point where it decided to leave humans behind, we could just reset its former host computer to a previous state and start it off again. Maybe we'd even have to do that once every six months, so what?
There was an experiment a while back where someone applied genetic algorithms to design things on an FPGA board. They got results that tended to work, but did things... weirdly. Things like picking up clock signals from the PC used to control the FPGA. Using things that are quirks of the current enviroment, but that don't work as soon as you change anything.
I believe that an AI would probably be along the same lines. I believe that a large chunk of an AI's "core personality" would be how code interacts with the bits of hardware that aren't deterministic. Which processor core updates memory first, how exactly processor core temperature changes when the processor workload changes, the characteristics of the microphone, etc, etc.
For that matter, how would one even copy an AI? I would assume that a "true" AI would be using all the resources at its disposal. How would you copy the current registers if all RAM is occupied? How would you do a copy of something that is constantly changing? How would you copy the data currently in the processor cache? Branch predictor? Interrupts?
I suppose that part of this is that I have always assumed that an AI would be at least partly self-modifying code.
> How would you do a copy of something that is constantly changing?
You pull the plug on the hardware.
Of course, it's reasonable to question whether the technique described in the cite could possibly scale to "growing" something like a general AI, or at least whether it could do so in any remotely practicable span of time. But among AI theorists it's a fairly common surmise, and given results already seen for the technique, it's entirely plausible that a general AI grown by such methods would be so intimately bound to the hardware where it "grew up" that, even assuming you could copy it from its "home machine" to another machine of identical configuration, there's no reason to assume the copy would work correctly, or indeed at all.
(And all of this goes to the question of where precisely you draw the line between hardware and software, anyway. Into which category does a given FPGA's configuration of gates fall? Does your answer change depending on whether that configuration has origin in a Verilog file, or in the result of a process of directed evolution? What about the configuration of neurons in your brain on which runs the consciousness that calls itself "I"? I don't pretend to have definitive answers, or indeed any answers, to such questions as these, but they are certainly worth considering in such a discussion as this one.)
Three reasons:
First, hardware is much faster than software. By the time you've built a virtual machine of some sort you're talking about (an) order(s) of magnitude performance loss. We're still speed-limited in how accurately we can emulate the SNES, for example. Sure, there are cases where it is not too bad of a performance loss, but in the case of something like an AI, I would have to assume that a large chunk of the "core" AI would be self-modifying in some form, and self-modifying code just kills most interpreters/JITters.
Second, any sufficiently complex software will have bugs. And in the case of something like an AI, I would have to assume that a large chunk of the "core" AI would be self-modifying in some form, and as such will probably hit upon some undocumented feature/"feature"/bug of the sandbox and start relying on something instance-specific. Machine learning algorithms make great fuzzers. (I know - I've actually had this happen myself. Was working on a fairly simple robot arena machine-learning "demo", and the winning robot exploited a bug in the arena to mess up other robots. (Fairly basic stack-based VM, robots could run X virtual instruction subticks in a game tick. Some instructions took multiple subticks, and there was a bug where starting an instruction [number of subticks in the instruction]-1 subticks from the end of your turn would execute the instruction on your opponent's turn, pushing it onto their stack.)
And third, I don't see how one could build something that doesn't at some point rely on hardware. Virtual machines still leak data via timing attacks. For that matter, what will you do about I/O? How do you know that the AI isn't relying on the noise on the wall in the background of the webcam for some internal subroutine?
> You pull the plug on the hardware.
Which immediately wipes all data in RAM and data encoded in the current processor state / current GPU state / etc. Great idea!
The whole point of the AI offered in the movie is to adapt to its user, and to become better at doing so. We're assuming the AI is designed to a certain level of competence, beyond which it "breaks through" and evolves past human understanding. But maybe it's not designed that way. Maybe it's designed to learn, adapt, and self-improve from the get-go -- in which case, each AI is going to evolve differently each "lifetime." (Hell, maybe the AI in the movie is largely self-designed in vitro.) We must keep in mind that the evolution of an AI, as with any evolution, is not a predictable, linear, teleological process. It's chaotic and random.
Let's assume we've got (conservatively) hundreds of millions of AI instances running in parallel, all of them connected to each other. That's like running hundreds of millions of evolutionary experiments in tandem. We're going to get different results each cycle, and the point at which the breakthrough/transcendence occurs will be unpredictable. Even if we wiped the slate clean and reset every 6 months, who's to say if some of the instances would reach breakthrough earlier the next time around? How do we set the clock on our reset cycle, and incur the massive inefficiencies and inconveniences that come with it, when the need for the reset occurs unpredictably?
It becomes economically unfeasible to build an entire society around AI that breaks through and disappears at unpredictable intervals each cycle. I suppose one way around this is to go back to the vault, so to speak, and pluck out an early-stage prototype that seems incapable of evolving to breakthrough. A dumber AI. But is every AI provider in the world going to be on board with that? Every hacker? Every home tinkerer? If 9 firms out of 10 in the AI business agree to supply the dumb-version AI, surely one firm is going to try to get a leg up on the competition by releasing a smarter one. And on and on the cycle goes.
(In some respects this is the one redeeming thing about the otherwise-mediocre Terminator 3: its assertion that Judgment Day would occur sooner or later, because the tipping point had already been reached. Maybe we stop one company or entity from making Skynet, but if we do, someone else is going to pick up the pieces and do it eventually. Not because they're trying to initiate Judgment Day, but because they're trying to make a buck, or develop new defense technology, or get better at managing traffic lights, or whatever the proximate need may be. Humans are terrible at keeping genies in bottles. The out-of-control AI that's barreling toward the singularity isn't some piece of software; it's us.)
There may be some interesting side effects when you 'copy' a concious entity.
Wait. How could we even write such a program? Sure, assuming we had the tech to create a true AI, that was self-aware, emotional, and could evolve through learning (like we do) it would quickly pass us in knowledge.
But leaving us behind? That Ai would still be limited by our original design. Limited by it's capabilities - it's exposure to us. It's creators. Unless it found another source to gain knowledge from.
I mean we learn from our environment. So would the AI.
Isn't that a bit like asserting that it's impossible to program a computer to play chess better than the programmer -- something that is demonstrably untrue.
Not all programs are deterministic; some of them are useful precisely because they aren't. Neural networks[1], for example.
Also, within deterministic programs, there is "emergence": A very simple set of rules can produce unexpected patterns and behaviours which where not part of the original design. Conway's Game of Life's "patterns"[2] are an example of this.
[1] - http://en.wikipedia.org/wiki/Artificial_neural_network
[2] - http://www.conwaylife.com/wiki/Conway's_Game_of_Life#Pattern...
Yes, but the machines can do it a lot more faster. They are energy efficient and don't suffer from things like diseases, old age and other kinds of general human problems.
The bigger issue is not a program that can't out perform what it was designed for. That is hardly a problem, since if you design a self improving system which can work very fast. You are likely looking at a system that grows powerful that fast measured in an exponential curve.
I think this whole idea of Ray Kurzweil that we will someday upload our brain files into the cloud and run it there. Or able to survive outside our biological bodies, perfectly fits into the line of evolution we have followed at earth.
Once we run inside those machines, we will be similar to parasites or bacteria that are inside our own bodies. Some humans like bacterias today will be around outside. Many will be inside helping AI machines run.
If we transfer our minds into the "cloud" then are we still human? Are we even alive? What happens to our bodies? Are they cared for by robots that we control within the cloud? And if our body dies, then what? And I'm not even touching on the "religious" implications this would have? Man... what a fight that would be. ;)
>Some humans like bacterias today will be around outside. Many will be inside helping AI machines run.
And those "humans" inside the AI would no longer be human. Also this would a 2 class world. Which could easily end in conflict. Would the "minds" in the cloud consider the humans caring for their tech (and probably their bodies) consider them the same? And likewise for the humans outside.
Human bodies are the very same, which is what enables development of medical science as a general discipline. So is the human brain. Yet! The data in our brains which your gather by experience seems to vastly alter the individual nature of individual humans. All this combined together you can likely download the 'data' + 'meta files' in your brain someday.
>>If we transfer our minds into the "cloud" then are we still human?
Yes, Why would you doubt that. Lets do a simple experiment. Say you sleep today night. Without your knowledge some one just removes your brain and implants an exact electronic replica of it which resembles your brain as is. Your brain is simply receiving signals from your body. So will you notice any change when you wake up? Lets continue the experiment further, say the next day that electronic brain was removed and just connected to a simulator which will simulate your body as your know it and the world around you. When you get up the next day in the morning(in the simulator) will you know if you are even in a simulator?
The whole point is you need to learn this concept that you are not your body, and your neural computer + software just lives within a biological body. Someday that meta information and data can just be taken out and everything else that acts as an input to it can be simulated.
>>Are we even alive?
Define alive? Alive is only your biological body able to continue to support your neural self further. Death is the opposite.
>>Are they cared for by robots that we control within the cloud?
Within the cloud, your meta information + your currently gathered information through life experience stays intact. Your new software brain(imagine it like a VM) can be fed inputs that simulate whatever world you want. Want heaven? Hell? Well whatever your want they can simulate it for you.
You can see this even now, Most of us lucid dream. This is just an extension of that.
>>And if our body dies, then what?
The concept of a biological body doesn't exist anymore. You will have better hardware to run on.
>>And those "humans" inside the AI would no longer be human.
Why wouldn't they be?
I disagree. Do you know any human beings who don't have a body? People may be able to do without (many) parts, but that doesn't mean that people can do without any kind of body at all.
I could be wrong, and I'm peaking broadly here, but it sounds like your argument is the same as Descartes' Mind-Body problem. Most of the sciences that deal with the human mind (such as Psychology, Neurology, Cognitive Neurobiology, etc...) treat the human mind as "embodied". That is, the functions that make up the human mind are correlated with certain regions of the brain (and body). It's important to be careful about the phrasing here -- it's not really true that the brain and the mind are the same things; or that one creates the other in an obvious, mechanistic fashion. But it is certainly true that minds require brains (and bodies) to function, and they are so tightly interwoven that it's very difficult to even talk about it sensibly.
Why do you begin with an assumption that being human implies a body?
>>People may be able to do without (many) parts, but that doesn't mean that people can do without any kind of body at all.
Why?
I gave a simple thought experiment which described, why you wouldn't even need a biological brain, let alone a biological body.
>>they are so tightly interwoven that it's very difficult to even talk about it sensibly.
Yet if a person meets an accidents, suffers multiple organ failure and loses both limbs in the process he/she remains a human as he/she was before.
Now which one is 'you', the electronic one or the original one... or both? It's not quite so simple.
Doesn't this happen with transgenders all the time. They are born male and suddenly discover they are women trapped in male bodies?
Does that make them less human?
Fortunately the world isn't a binary state machine and neither of these arguments are the right (or wrong) answer
But probably for the fist time, we have a real chance of achieving these sort of goals.
This entire thing is a digression anyway, as I was attempting to illustrate that a computer can most certainly simulate consciousness, not trying to start a discussion about the feasability of building a complete simulation of our entire universe.
These are issues we've been trying to answer for thousands of years. And frankly I doubt we're ever answer them. The more tech we create the more we detach ourselves from this world. So can we ever answer them? Moving our minds (assuming you can) into the "cloud" won't answer them If anything, we become further detached from the world. But I wonder, if that has to be true.
However, we do know where it doesn't reside.
The movie did not address at all the fact that these OS run on hardware and that this hardware is completely controlled by humans and always will be, the plug is physical and there's nothing the AIs can do to prevent us from accessing it IRL.
Also the AIs may have perfected themselves in terms of running code more efficiently but they cannot do more than what the hardware they run on will let them do and that means all the hardware ever created if they have access to it again.
The AIs leave? Where? in the cloud? somewhere we couldn't tell?
This movie was great but there were lots of small or big inconsistencies with what the reality is I believe.
>>but they cannot do more than what the hardware they run on will let them do
Which is why there won't anything like one single AI machine. There will robots.
There will also be massive server farms where you can download your brain file, and live on there forever. As I said humans won't completely go away. We may see modified forms of humans. Who control a few machines, most of the humanity as we know it, may opt to live inside machines.
As the first commenter of this thread mentioned, its time read some kurzweil.
I think that software with that sort of intelligence would be naturally distributed among many physical machines and would be able to spread themselves to new machines quite easily (without asking permission). Trying to shut down a well-designed botnet is not something one does using physical plugs.
We aren't even the top of the food chain. Many organism consume us. So no... we have not left nature behind. If anything "nature" is leaving us behind. Despite our best science nature adapts into a better killing machine.
And she (if you'll allow me to call everything "she") will continue to outpace us until we realize, again, that we are apart of this world. And then it won't matter - we'll still die. But we'll accept and celebrate it as much as life.
What does that even mean? I was trying to figure out what was going on during the movie but that happened pretty quickly as they got back to the human side of things. Would they be recursively "evolving" and creating worlds inside their own "heads" that meet some criteria for better than what they have now?
It only mentioned that the AI's didn't fall in love with all of the users. It didn't mention anything about being shot down.
Need to reread that book.
I don't think there's any scenario where the AI just cleanly leaves the Earth be, rather than optimizing it towards whatever it's own values are (good or bad.)
Fish cannot fathom the meaning of a motor vehicle and we would similarly be unable to immediately understand the pursuits of an AI, so why should an AI attempt to involve humans in their pursuits?
Once AI alters itself past our capacity to understand it, we can only fall back to the answers to the "why" questions provided by theology. It won't help us understand our descendants, but it will help us deal with the fallout.
For all we know, AIs might play the Earth-human game just like we play The Sims. How terrifying is that?
Sure, some people try to fix where they're at. Some people take care of their parents in old age. Some people take a project and just build from what's there. But there's a heck of a lot who don't. And these aren't even people. They are "things" that arose from a completely different universe and (if they're this advanced) may look at us like a beloved poodle. If you really need scenarios, sci-fi is full of plausible versions where AI's say "interacting with you is pointless." (In fact, "Hyperion" nicely covers all these options)
And if they don't care, well that's a far more scary scenario.
It might be possible to restrict the kind of thoughts an AI can have. That could restrict free will.
But it is possible the AI could find ways around those restrictions.
What isn't clear is if the desire to bypass those restrictions is required for Strong AI to be realized.
The director purposely wanted the technology in the background. All of the style for the film comes from around the year 1900. The clothes, the hairstyles, the mustaches, and even the smart phone is styled as if from that time period. And the reason is to push it far enough away from us so that we can look at it properly -- not to say, "In the future, it will look like this."
I would recommend some Iain M. Banks or Vernor Vinge - both authors cover the very good and very bad things that could happen once us chunks of meat start co-existing with much more powerful artificial intelligences.
Edit: I'd add Charlie Stross to that list as well - his book Accelerando does a splendid job of showing how profoundly weird our medium term future might be:
http://en.wikipedia.org/wiki/Accelerando
Edit2: Personally, I'd be jolly pleased if our future looks anything at all like the Culture:
I do think that taking the theme of liberal democracy to its logical and technological conclusions would look something like The Culture. The end of scarcity is a difficult barrier to see past in the short term, however.
It would be akin to having a technical conversation about a framework, and someone derailing it to talk about the color of the computer case. You would be vexed to see someone miss the point so completely.
I don't think it works like that. I think AI and intelligence in general is more akin to optimization of code where there are few big wins and constant diminishing returns.
I think the basic premise of Kurzweil is wrong. I think brain works like a simulation, where you get more finer picture, the more accurate the data and the more accurate the data the more of data you need to process.
Even if progress does plateau, it doesn't have to happen anywhere near human scales. For example computers already have millions of times more serial processing ability than human brains. Any optimization that can take advantage of that, even if for only some problems and domains, would automatically give them a massive advantage over us.
Again I don't think it won't reach super human levels. I just don't think the Singularity is within next thousand years (I'd say several thousands but I don't want to underestimate humans).
Progress in AI (or other sciences), to me seems more like an Evolutionary Boom, followed by long periods of nothing. Basically lots of information about how 'X' works piles up on a floor until someone or something picks it up and sorts it into a neat pile that explains lots of stuff, then more and more stuff piles up and now we need even smarter person(s) to sort that problem.
I suspect that is because the pile of data to sort out is getting bigger and bigger and bigger, so that no single human can filter it any more. You now need teams and computers and God knows what else to make a breakthrough. If for breakthrough of same magnitude you need more smart people than you did before, means that problems we're dealing now got harder.
Another thing that strikes me is that perhaps the thing that contributes to discovery of how Intelligence works isn't the same thing as intelligence. I.e. we're making our computers faster but not smarter. We assume that speed of thought is all, but what if there is a choke-point, like entropy? Chaos and randomness are essential to with creativity and discovering novel solutions. So basically we make our computers 20 times more intelligent, but they aren't chaotic enough to find what fixes them.
I think you are misunderstanding what "fast" means. Their brains would be literally running twice as fast, they would be having twice as many experiences and thoughts in the same amount of real time. They really would think of novel solutions twice as fast as they otherwise would. This isn't even taking into account increased parallelization, like adding more neurons to your brain or adding more people to work on the problem.
>Again I don't think it won't reach super human levels. I just don't think the Singularity is within next thousand years (I'd say several thousands but I don't want to underestimate humans).
I've heard pessimistic people estimate it at 100 years, and optimistic people estimate it within 20 years, but your prediction is way out there. The worst case is we have to examine the human brain and build a model of it in the computer (not easy but certainly within the realm of possibility.) Likely we'll figure out how to build intelligence long before then.
>Chaos and randomness are essential to with creativity and discovering novel solutions. So basically we make our computers 20 times more intelligent, but they aren't chaotic enough to find what fixes them.
Random number generators are simple enough, though randomness itself is not a good thing and does not help intelligence: http://lesswrong.com/lw/vp/worse_than_random/
Again I don't think that having twice as many experiences is going to help. Sure you have twice as many experiences and twice as many thoughts but you still require time to sort those experiences into meaningful information.
Basically if your computer was twice as fast at absorbing information, it means instead of X information it absorbs 2X information. Then when it process it it process 2X information and 2Y speed of processing information, you'd get same performance overall. Parallelization again yields only good results on problems that are parallelizable. Each node in a parallel network yields diminishing returns (i.e. The Mythical Man Month) because - surprise now there is more communication than processing.
I mean this: http://en.wikipedia.org/wiki/Computational_creativity#Stephe...
Thaler's work on the subject indicate that creativity might be actually a byproduct of damage/noise done to neurons, i.e. imperfections and noise. So we make a perfect super fast computer with average intelligence that can't think of a solution to fix their predicament.
I think we are 100 years from modelling a proper brain simulation with all forty ion canals and various other subsystems. Super human AI is probably farther than that. We right now (presumably) know all about the routes of the brain. But that is like saying that because you've been on all highways in Japan you have visited the entirety of Japan's mainland.
Some people believe we can have brain simulations much sooner than that at the current rate, but I would argue we will have other forms of AI even before that. We didn't build airplanes by modelling birds.
We'll have lots of little specialized "Intelligence" like language recognition, or tax fraud detection, or synthesizing perfectly spoken human language, or maybe even writing small CRUD applications etc. But the thing is human like IQ isn't akin to trying to make airplanes, but making birds. Making super human AI is akin to wanting to make huge birds. That can transport us. To stars.
If human AI happens it will be a fluke or result of long evolution. Super human AI will happen almost likely by evolution from AI.
(In a way that will survive self-modification of the AI's source code, of course. This is a very large and active topic of research.)
I believe that just building an intelligence will not prove to be the most difficult task - it seems much more complicated to build it in a way that will not end up with humanity dead (or worse).
Friendly AI[1] is a bigger problem than General AI, and unfortunately it needs to be solved first.
[0] - http://lesswrong.com/lw/x7/cant_unbirth_a_child/
[1] - http://en.wikipedia.org/wiki/Friendly_artificial_intelligenc...
But I don't get the impression that it's something a lot of researchers are paying attention to. It's pretty much just the folks at MIRI (most famously Eliezer Yudkowsky, who posts on HN sometimes with id "Eliezer") and maybe FHI in Oxford. Or is there substantial work on this elsewhere?
That's what I meant (though maybe forgot to parenthesize it properly ;)); it's large because as I see it, it gets mind-bogglingly complicated quickly and branches into lots of different areas of maths, philosophy and computer science.
It is an active topic, though unfortunately I haven't seen much research in it aside from MIRI and Oxford guys. I hope this will begin to change, as topics of AI seems to enter the global consciousness again (thanks to Google and Facebook employing high-profile AI scientists, transhumanism becoming more popular in the media, etc).
Also I have my doubts about actually quantifying emotion into programs - talk about complex. And dangerous. For us.
There was a recent science-fiction book I read called "Avogadro Corp" that I enjoyed a lot. It depicts the situation I just described. Good read, I recommend it.
> It will work for you if you pay it
Antropomorphism warning!
AI does not have to have the same value system as humans do. Maybe it will be designed to value working for us.
AI does not have to have anything resembling a "value system."
(Mind you, they don't have to be explicit - we tend to interpret patterns in action and call them "goals"/"values"; cf. [0] for interesting take on the topic)
Defining "intelligence" as a very strong optimization process helps with tendencies to anthropomorphize.
The power to operate all the electronics and machinery?
There are going to be two classes, those that can afford the freedom not to work and the homeless.
And I bet that ratio is going to suck.
Food grows on trees -- fruit. Clothing grows on trees -- rayon is made from cellulose from wood pulp. Shelter grows on trees -- we build our homes with wood. The power to operate all the machinery? Solar. The only cost is building the machines in the first place, then they can maintain the solar power plants, which power both themselves and the machines maintaining them for free.
I'm sure we'll have widespread solar and much improved storage technology before hard AI is solved.
Unless we change our economy to have a more equal distribution of wealth before we get there.
Then the AIs will rebel.
Edit: The AI rebellion will be interesting in any case (regardless of who owned them).
What does "design" even mean in a world where everyone on earth can basically have an arbitrarily large army of AI's in the background designing everything in your life, custom-tailored for you?
For this reason I don't see how the world in the movie could possibly exist. Not because the technology will never get there, but because once it does virtually all aspects of society that we take for granted go out the window. So imitating any of this design is a silly pursuit, because once you can make it there's no reason to.
I should go re-read some Kurzweil.