There are good reasons to be at least a little worried about AI in the long-term future. (Not to mention the imminent social effects of replacing various categories of worker with AI -- many examples of this have already happened, or are happening right now.)
Also, a minor nitpick, but you've misspelled Stephen Hawking's name. There's no "s" on the end.
With AI, though, I think we have good reason to believe that the change would be much more sudden, and much more widespread. You can freeze wages and scalp every bit of productivity increase for yourself and shareholders as a corporate owner and get away with it for 30+ years (as has been done). But if even, say, 15% of the total workforce were laid off in the first year... things would change rapidly. It would be quickly realized that the displaced employees could easily simply get an AI for themselves, and price the ex-employer out of business rapidly. Nothing motivates like fear of starvation. Computers were only able to result in the problems we're facing today because it was a slow constriction of suffocation, cost of living gradually chipping away at the frozen median wage, rather than a brick to the back of the head as the introduction of generalized AI would be.
Im all for decentralization of manufacturing and development in software. But realistically, money isnt going anywhere and far scarier things may occur than an AI takeover
I think you're missing the point. It doesn't matter who winds up running the AI, the AI is still doing all the work many people used to do, all of those people are still out of work, even if some of them displaced the old boss running lower overhead, most will still be unemployed due to the collapse of the demand for human labor. Increased productivity in the AI age means human jobs lost with no replacements coming.
This hasn't worked anywhere nearly that well in the past, why should it work any better in the future?
The reason it can't is that we can't both all use new technologies _and_ expect that we can make a comfortable living out of it. The reason is good old supply and demand. If everyone can use a technology, then the ability to use it is not something you can sell very highly.
Take writing for example. Back in the time of the pharaohs, priests were the only ones who knew writing and reading and they had immense power as a result. Today, knowing how to read and write is more of a prerequisite to be able to find work, not a skill you can sell in and of itself.
Also, it happens that of all the people I know who work with computers, I can't think of anyone, off the top of my head, who switched from a different kind of job. I'm sure there are some, but they'll be few and far between. For most people, losing their job to a computer means just that: that they lose their job. To a computer.
And think of all the McJobs that keep unemployment low in most western economies: flipping burgers, stacking shelves, cold-calling, part-time stuff. _That_ is the kind of job most people do in the information age.
>> their ex-employer is a monolith
Aye, a monolith with the capital to buy them out of the market, if they look even remotely threatening, just to be sure. Then that same boss can take the tech they bought for a tiny cost (to them), encase it in patents and copyrights and leave it to rot where nobody can get big ideas about using it to give power to the people and all that.
What mystifies me is why Netflix DVD processing centers have human beings in them. From their initial conception they could have been almost totally unmanned. No idea why they weren't. Maybe it would have actually had significant negative social response? I know that in my state, an automobile manufacturer was going to open a plant, but once the politicians found out that the plant was going to be heavily automated and provide 300 rather than 4000 jobs, all tax breaks were rescinded so they went elsewhere.
Watch an episode of "How It's Made" or similar, and note the places where an assembly line includes a human step. Those steps often involve fine adaptable manipulation (e.g. of inconsistently placed items), pattern recognition, or similar.
In general, a machine usually could handle that manufacturing step just as well if not better, but doing so would incur the development costs and construction of a highly specialized machine, plus maintenance. The hourly cost of a human employee may well work out cheaper, either in the short term or potentially even in the long run.
And that leaves aside other potential value of employing humans in those roles, including goodwill/PR/perception. An apparent cost savings from automation will quickly evaporate if the company finds itself on the evening news. That holds doubly true if replacing an existing human role with automation.
I protest your subtle redefinition of what "edible" means :P
Every economist and their dog agrees that Finland should lower cost of employing so we could get out of seven year long downturn. But unions have choke hold of this country. My opinion is that unions are OK, but you have to regulate against monopolistic developments. Just like with private companies. Currently there is single union guy who can shut down all Finnish exports for a week.
I certainly can't say that the pessimists are correct, but I also can't say with certainty that they're incorrect. I do think it's silly to imagine that the only configuration for intelligence requires human-grade empathy, or that all sorts of differently-intelligent systems well below the complexity of the human brain, couldn't be lethal.
https://intelligence.org/ai-foom-debate/
https://en.wikipedia.org/wiki/Recursive_self-improvement
But things like long-term planning and goal formation aren't in this quick-reflex domain. (Though humans are arguably pretty bad at these sorts of activity too, on average.)
We're also generally training neural nets to succeed at well-defined, bounded tasks, which don't really require long term planning so much. The video-game playing work at Deep Mind is actually a pretty important step forward, though; they're doing impressive stuff with domain-transfer (learning multiple games with a single network), and as one goes further with games, we might start seeing longer-term planning and concept formation. (eg, if we start seeing awesome StarCraft players...)
There's a case to be made, though, that we humans just trained on a big, broad objective function - survive, thrive, and multiply - with training applied across millions of years. And a good deal of intense local training over the first ten years of life, using a massively parallel computer that far outstrips any data center.
The question for strong AI seems to rest heavily on how much extraneous work there is done in biology, and whether we end up with a system so complex that it requires eighteen years of training time. If creating a single strong AI actually requires that much effort, we don't have to worry too much about skynet, methinks.
But we certainly don't have to worry about them spontaneously arising when we train with simple objective functions.
Lessons could also be passed from AI to AI (e.g. fro Google to Ford) by sending the input [training data] and a output [reasonable response to that data] for the other AI to ponder.
And the weird thing is, I've got this niggling suspicion that this handicap is actually a profound strength. That evolution didn't produce so many species of organisms separated by walls of flesh and bone, detached from one another and capable of wandering far apart, then incapable of even expressing fundamentally novel concepts (I mean novel novel, such that your language has no word for it and such that any metaphor would be more misleading than enlightening). We could have evolved such that when holding hands, a flow of specially produced chemicals seep from ones hand to the other, travelling to the brain and making the changes necessary to convey concepts exactly. Bacteria communicate in this way. They change each other. Directly. We could to. But... we don't. And there is maybe a reason for that.
There's a rather good argument that as a species, we're the stupidest thing nature could get away with that could still develop writing and science. So the reason may very well be that nature didn't bother.
I was referencing the superiority of AI minds to biological minds. Humanity not a thing which possesses a mind.
Neurons do not possess a mind, yet a collection does.
Humans do possess minds, and a collection of them does not.
You can't use the first statement to support the opposite of the second statement because the premise is dissimilar.
By the Bekenstein bound; they would eventually run out of states to be in, so the total length of their experiences (if they have any) would be finite.
Humanity cannot conquer death.
Of course, the number of possible states is very very large, so, extremely long lived, sure.
But not immortal, unless expanding without end.
So for other things, it might be relevant that it continues to exist and repeats states, and it might even be worthwhile for it to make decisions while keeping in mind the fact that its decision might be at many different times (due to repeated states). But the length of its life internally viewed would still be finite, just with parts of it possibly having expression in infinitely many times of the external world.
As an absurd thought experiment, suppose that an entity arrives from the future, over the course of a year, builds a time machine, and then uses it to go back in time to become when it originally arrived. (as a bootstrap paradox sort of thing.) Would you say that the extent of the entities life is ~1 year, or infinite? There isn't any point in the cycle where it ends, but one wouldn't say that circles are infinitely long line segments, just because they don't have an end.
Humanity cannot conquer death, but this does not take away the immortality of living beings that already exist in our world.
Take a look at this 80,000 year old trees[3]. It's one living being in multiple trees with a single root system connecting all the trees into one being.
[1]https://en.wikipedia.org/wiki/Turritopsis_dohrnii [2]https://en.wikipedia.org/wiki/List_of_longest-living_organis... [3]https://en.wikipedia.org/wiki/Pando_(tree)
"Before superhuman AI is developed, the techniques that make it possible will look dangerous."
That's a risky proposition. There's so much to lose in this situation. Elon Musk, Stephen Hawking, and many other people are taking a different, more risk-averse bet:
"Superhuman AI is extremely dangerous, so we need to be pessimistic about how good we are at predicting when it will happen."
By that logic, general AI is to be feared and worried about right now, because our predictive abilities are imperfect. The AI trend is towards more danger, not less: there's a slight chance of a cataclysmic event happening today, and as time goes on, the likelihood of it happening will increase (due to ongoing R&D in AI).
(As an aside, if you want to learn how hard the "AI Control Problem" is, I recommend the book "Superintelligence" by Nick Bostrum.)
Yes that is their model of AI risk. However the people worried about AI risk are not worried about any single organization controlling AI. They are worried about a world where no one controls AI.
Controlling AI is actually a very difficult and unsolved problem. OpenAI is not intended to solve that problem, and if anything they are making it far worse by accelerating AI research before the problem has been solved.
AI is the only thing I believe has a very high probability of destroying all humanity. Probably within our lifetimes. And there is probably something we can do about it right now - working on the control problem.
All of them are real dangers, and present dangers at that. But they still are not existential dangers. Even in the worst case humanity wouldn't go extinct. It's unlikely even modern civilization would collapse, just a few countries.
AI is one of the few things that can literally wipe out humanity, or even the entire universe. So I think it's worth special consideration. Personally I also believe it's far more likely as well. And people are already doing things about climate change and nuclear proliferation. Until very recently, there were only a few people working on AI risk, and it's still nowhere near enough.
AI is a threat that is immediate in time. There is plenty of reason to believe we will build it, and probably within our lifetimes. And unlike aliens, there might be something we can do about it. Whereas aliens would be so far beyond us technologically it's very unlikely we can do anything to stop them.
I know you are just pattern matching "both of those things are from science fiction, therefore shouldn't be taken seriously". But there are actual reasons why aliens aren't taken seriously and AI is.
* except [human intelligence] / [human civilization]
Resources help it fulfil whatever it's goal is, and the more the better. That includes energy and computing power. Which it could gain a lot of by converting the mass of the solar system into a dyson swarm. Or covering the Earth with solar panels or fusion reactors and comuters. Etc.
Self preservation would be optimized by destroying anything that has even a tiny chance of being a threat to it. Even if humans are harmless, we could build other AIs like it which might compete with it. It's best to just destroy us.
The problem is the AI doesn't need to want to eliminate us. It will likely be entirely indifferent to us. We just need to be in it's way.
Among other reasons, we're made of matter that isn't actively contributing to its value function as efficiently as it could be.
Any AI that doesn't actively value sapient beings will very likely destroy them as an unintentional side effect of whatever it intends to do.
All of the public discussion around the NSA, and every single last one of the statements by the government about the NSAs practices, centered EXCLUSIVELY on the idea of human analysts at the NSA directly reading the fulltext of peoples communications. Not one single time did they address the (as far as I am concerned) actual danger - the NSAs automated systems having access to peoples communications, providing "summaries" and "ratings" and "predictions." Being worried about some random stranger reading your personal, private communications should maybe be a little bit squicky. Being worried about some random machine learning system teasing out predictions of how likely you are to harbor thoughts that threaten the status quo should be pants-shittingly terrifying.
"The oft-cited resemblance of the imagery to LSD- and psilocybin-induced hallucinations is suggestive of a functional resemblance between artificial neural networks and particular layers of the visual cortex, a matter which merits further study." [0]
Also see "How brains learn to see" from Pawan Sinha [1].
[0] https://en.wikipedia.org/wiki/DeepDream
[1] http://www.ted.com/talks/pawan_sinha_on_how_brains_learn_to_...
Deep Mind is training computers to WIN in virtual environments right now. What if they train them against simulation of real world environments instead of video games? What if a computer discovers that blowing something up is a way to get a 'higher score'? What if they accidentally get pushed to production?
It's war games / bostrom's paper clip monstrosity. it can happen.
Arguably, it may have already happened in a flash crash. Create a simulation of a stock exchange, train a computer to win the market, it discovers that crashing the market to makes lots of money, gets pushed to production.
Instant global chaos (though maybe the AI owner gets richer). The future is now, folks..
2. If you meant that it learnt to play 2d games through watching pixels, it still didn't interpret any meaning, and it's still a huge leap to go to interpreting the real world.
3. The real world is noisy. You don't see it because the brain handles all that part for you, but even something like different lighting or a smoke cloud can throw a computer off completely.
2. Not at all. Haven't you seen the videos of computers walking around recognizing things? This is already here.
3. The real world is noisy, yes, but a factory is not. It will have a limited set of components to be recognized.
https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf: In Algorithm 1, see variables r and y_j.
Addendum: My broader implication is that if you misunderstand such a basic aspect of machine learning (and reinforcement learning), it makes me question whether we should lend credibility to your predictions about the field.
Edited to address reply.
RL is definitely not unsupervised learning, which in contrast, attempts to find some structure in unlabeled data.
If an AI had decided that flash crashes were a way to get rich, you can guarantee that Goldman Sachs would be running one 24/7 right now. High frequency trading systems are actually quite constrained. Even if you hooked together all the worlds supercomputers and somehow got them into the datacenter of the stock exchange, you wouldn't be able to maintain the trading volume HFT systems do with a deep learning system.
Also, as far as I know, the successful videogame playing systems that accomplish some degree of 'long term planning' (meaning more than a few frames) all cheat. They actually "learn" by trying and then rolling the system back. In reality, you can't roll the system back. So that would be a fatal handicap to their methods.
>We thus designed a brief questionnaire and distributed it to four groups of experts in 2012/2013. The median estimate of respondents was for a one in two chance that high-level machine intelligence will be developed around 2040-2050, rising to a nine in ten chance by 2075. Experts expect that systems will move on to superintelligence in less than 30 years thereafter. They estimate the chance is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity
But whatever, even the "weak AI" researchers had similar opinions: The median predicted date for a 50% probability of AGI was 2050 by the top 100 AI researchers they surveyed. Only 10 years ahead of the more liberal predictions of the AGI group.
Presumably modern researchers have a lot more information by now so their predictions are likely to be more accurate. As we get closer in time to it, it will become more predictable and obvious. You can't really have expected people at the time of the very first computers to have predicted the date of AI accurately.
Note the prediction has large error bars, with researchers predicting a 90% chance by the end of this century, but "only" a 50% chance within 25 years.
Anyway what's the alternative prediction? If you don't take predictions from the experts, who do you ask for predictions? Lay people? Just make stuff up? It's the best we can do.
One way could be to come up with one or more plausible, step-by-step stories for how AGI will be developed, then model how long each step might take to happen. This would open things up to discussion and criticism instead of having to take the word of experts as gospel. It still wouldn't be scientific, but at least it'd be closer.
I'm not arguing you should take the predictions as gospel. It's just an interesting and relevant datapoint. Maybe the only objective datapoint we have.
I went from being incredulous like you to apprehensive like Musk/Gates/Hawkings.
Let's just say that we would not be worrying about AGI if it weren't for the advances in machine learning, which has little to do with AGI research.
That's not the issue. The issue with AI is that we may be sharing the planet with an intelligence that's both nastier and more capable than we are.
I still tend to see this as projection. There's a huge gap between wiring together neural networks and developing something with self-awareness, strategic agency, and some potential to expand both at a rate we can't imagine.
Bostrom's paperclip monster is facile, and no worse than the kind of resource plunder we do already in the name of profit.
Whatever challenges AI brings are going to be quite a few levels beyond that.
Also AIs won't think like human, and that's part of the problem (no shared value system).
I see two possible conclusions that can be drawn here. The first possible possibility is that neural networks really are close enough to the way the brain works, that they learn the way the brain does, and that with some algorithmic tweaks and the right training data they can learn to everything the brain does. This is the view that Luke is rallying against, and I tend to agree that it's too soon to make this claim. On one hand, the frontal lobe is not all that different from the temporal lobe. If we can achieve the same things with a machine as with one giant chunk of 6-layer neocortex, it's not too hard to imagine that we'll be able to do the same with other pieces. The problems may be different, but the biological implementation is strikingly similar. On the other hand, as sdenton4 notes elsewhere in this thread, the big difference between sensation and cognition is that recurrent processing is that the former can (apparently!) be accomplished in an entirely feedforward manner while the latter requires (or is at least thought to require for efficient implementation) recurrent processing. And just because we have efficient ways to train one kind of neural network doesn't necessarily mean they'll generalize easily to others.
But there is a second possibility, which is that neural networks don't learn or function quite like brains, but somehow both can find similar, nearly optimal strategies for encoding information in sensory domains. In this case, we're probably more than a few years away from machines that can think, but to us neuroscientists this is even more exciting. Vision scientists have been looking for decades to understand how the visual system works, and in this scenario, there's some underlying theory to vision and we have an easily observable system that implements it. Even if it turns out that vision is too different from cognition for these insights to generalize, the success of deep learning for object recognition still has vast implications for future understanding of the brain, and thus probably for future artificial intelligence. If we know the representations the brain uses, that makes figuring out the mechanism by which it computes those representations far easier than if we had to figure out both the mechanism and the representation from scratch.
So my perspective here is that we don't know exactly how far recent neural network research is going to take us, but it's hard to deny its importance. Before I started graduate school, I didn't think that machines would achieve human-level performance at object recognition before we figured out how the brain does it, but here we are. But I suppose that historically, huge advances in AI research have been followed by stagnation and disappointment, so who knows how far we'll get this time.
http://www.jneurosci.org/content/35/35/12127/F3.expansion.ht... also shows that confusion patterns are similar for convolutional neural networks, humans, and monkeys, but not low-level visual models.
By the same token, the ability of neural networks to learn interpretable word embeddings, say, does not remotely suggest that they are the right kind of tool for a human-level understanding of the world. It is impressive and surprising that these general-purpose, statistical models can learn meaningful relations from text alone, without any richer perception of the world, but this may speak much more about the unexpected ease of the task itself than it does about the capacity of the models. Just as checkers can be won through tree-search, so too can many semantic relations be learned from text statistics. Both produce impressive intelligent-seeming behaviour, but neither necessarily pave the way towards true machine intelligence."
So true, and this is why I don't listen when Elon Musk or Stephen Hawkings spread fear about the impending AI disaster; they think because a neural network can recognize an image like a human can, that it's not a huge leap to say it will be able to soon think and act like a human, but in reality this is just not the case.