Specialized AI (and I hate calling it AI) is coming along really quickly. We're getting better at it in existing fields and learning to apply it to new fields. More than anything, we just have so much data on everything now, and computers are pretty powerful now, so even old school models are finding tons of new applications.
Generalized AI is a different story. We are a few really major breakthroughs away. We aren't even 100% sure if they are possible, much less understanding how to do them. These aren't the normal slowly-chip-away-at-it breakthroughs, these are things we have no clue about. With something like that, who can really say how far we are? It could be 5 years, it could be never.
On the other hand, once someone cracks that, a huge number of low-end jobs will be automated.
Deep learning is still mere perception. It doesn't handle memory or processing, it just transforms input into output, typically trained by Big Data, way bigger than necessary statistically speaking, given the world we live in.
AGI requires super aggressive unsupervised learning in recurrent networks, likely with specialized subsystems for episodic and procedural memory, as well as systems that condense knowledge down to layers of the network that are closer to the inputs. At a minimum. And nobody is really working on any of that yet (or at least succeeding) because it's really damn hard.
That's why everyone in "AI" is rebranding as a deep learning expert, even though deep learning is really just 1980s algos on 2016 hardware - you gotta sex up feed forward backprop or you don't get paid.
Edit: to be fair, robot control is much simpler than AGI, and might be mostly solved with deep learning somewhat soon, I forgot the context of your post.
There's more going on than convolutional neural nets. Architectures with memory and attention mechanisms do exist.
As examples: https://arxiv.org/abs/1506.03134 https://deepmind.com/blog/differentiable-neural-computers/
What really doesn't exist is any meaningful stab at unsupervised (or self-supervised) training on completely unstructured inputs or any sort of knowledge condensation/compression, at least for time dependent problems. These are of paramount importance to the way we think, and to what we can do.
There's a lot of trivially low hanging fruit, too - I still have yet to see even a grad school thesis that starts with an N+M node recurrent network and trains an N node subnetwork to match the outputs based on fuzzed ins, and then backs that out into an unsupervised learning rule that's applicable to multiple problems. Or better, a layered network that is recurrent but striated, that tries to push weights towards the lower layers while reproducing the same outputs (hell, even with a FF network this would be an interesting problem to solve if it was unsupervised). These are straightforward problems that would open up new avenues of research if good methods were found, but are mostly unexplored right now.
I could be wrong, if I had real confidence that we were close I'd be working on this stuff, but I'm collecting a paycheck doing web dev instead...
Differentiable neural computers - https://deepmind.com/blog/differentiable-neural-computers/
> we introduce a form of memory-augmented neural network called a differentiable neural computer, and show that it can learn to use its memory to answer questions about complex, structured data
So it seems that deep neural nets can have memory mechanisms and be trained to solve symbolic operations.
A few weeks ago there was a paper posted on "Synthetic Gradients" which should make it much more practical to train RNNs for games. Before it required saving every single computation the computer makes to memory, which uses a huge amount of memory and computation. Using synthetic gradients they need only store a few steps in the past. And it can learn online.
And have some problems with PacMan as the system can't plan: https://www.technologyreview.com/s/535446/googles-ai-masters...
Maybe those should be the other way around?
AI problems can be characterised as those where there's no clear path to a solution (otherwise we just call it "programming"); tackling them necessarily involves trial-and-error, backtracking, etc.
Since there are far too many possibilities to enumerate, solving such problems requires reasoning about the domain, e.g. finding representations which are smooth enough to allow gradient descent (or even exact derivatives); finding general patterns which will apply to unseen data; finding rules which facilitate long chains of deduction; etc.
The difficulty is that there's usually a tradeoff between the capability/expressiveness of a system, and how much it can be reasoned about. If we choose a domain powerful enough to represent "the field of specialised AI generation", for example turing machines or neural networks, methods like deduction, pattern-finding, gradient following, etc. get less and less applicable and we end up relying more on brute-force.
To me, this is where the AI breakthroughs are lurking. For example, discovering a representation for arbitrary programs which allows a meaningful form of gradient descent to be used, without degenerating into million-dimensional white noise; or to take deductive knowledge regarding one program and cheaply "patch" it to apply to another; and so on.
There was a great article recently on HN that highlights the current problems:
http://www.theverge.com/2016/10/10/13224930/ai-deep-learning...
https://news.ycombinator.com/item?id=12684417
Just because we may acquire the processing power estimated to be used in the brain (in operations per second) doesn't mean we know how to write the software to accomplish the task. It is very clear current algorithms won't cut it.
Also, I think we are a few orders of magnitude off on raw processing requirements because I think it is a bandwidth issue as much as an operations per second issue.
TL;DR - you could throw as much processing power and data as you want at any current deep NN or their derivatives and you wouldn't get general intelligence.
That said I don't think the winter will be as bad as before because, like OP says, specialized AI is useful.
I think a lot of the early AI research (not my specialty) had the idea that if we made a bunch of systems that were good at their own piece of the puzzle, then we could just tack them together and get real intelligence. It just didn't turn out that way. Something I'm more familiar with is graphical models, and while they in principal could do amazing things when you stick little expert components together, we've proved the complexity grows pretty badly in the most general cases that would have been really amazing. I'd bet similar things happened in other "let's put a bunch of specialized systems together" tracks. Maybe we can do it, but not the naive way that would have been great.
Then you can get interesting and philosophical about it, where you might even say that emulating intelligence and intelligence are different. Like the chinese room thing, or even a character in a story vs a physical person. I'd rather not weigh in on that right now, but there are good interesting arguments both ways.
This would be a very surprising result. For example, if I can make a TSP-solver-emulator... I have a TSP solver.
General AI - think about thinking machines
We can do the former, have no clue how to do the latter.
This theory compactly explains why no one knows how to do general AI.
I don't believe there is a special sauce waiting to be discovered.
Many humans go their whole lives without doing that, so I'm aware it's a high bar. But it's a bar that some humans do pass, and if AI is to be more than just a helpful gimmick, it'll have to do that as well, since I'd like to believe all humans have that potential, even if not always realized.
(Obviously a helpful gimmick still does have value.)
Looking at the brain, it often does look like a bunch of interconnected specializes neural networks.
And computers are fast now. And RAM is infinite. And GPUs are fast and plentiful. And all of this is cheap.
AI stuff is already real more than most of us realize on a day-to-day basis. While machines might not be "intelligent" per se, their cleverness has definitely started impacting desk jobs.
The desk job stuff is what most of those alarmist news items are worried about. Nobody seems to worry [too much] about reducing blue collar jobs through automation. Can you imagine how many people you'd need to unload a modern container ship without computers tracking stuff and optimizing storage? Or how much work it would take to harvest modern crops where automated harvesters are used?
Imagine a human who lacks empathy entirely. That's a disability. They may be able to do some amount of destruction, but at some scale they simply lack the social intelligence necessary to compete with the entire species.
This is the most common mistake I think people make when reasoning about AI: they think human limitations are weaknesses. But they're not weaknesses they're tradeoffs. Natural selection has had a chance to reward all kinds of variations, including more cortex, and less empathy. But we ended up where we are because of tradeoffs.
Any AI which is intelligent in the same way humans are will also have our limitations. Any AI which doesn't have our limitations won't be as smart as us in those respects.
You have to really ask yourself what the difference is between a human with an AI simulator and an AI with a human simulator. In the limit of simulator quality there is none.
So long as AI remains tasked with categorizing human-taken photos or playing human-created games (no matter how "complex"), AI will remain just that -- artificial -- and not "real".
Given the rate of hype that you point out, I suspect a winter of some kind will hit before enough folks realize this.
[edited: reorder]
By the way, the above requirement of real world data probably even applies to building chatbots. (A successful chatbot will need some understanding of the world it is talking about; this is what researchers mean when they say "grounding" is important for NLP).
I suspect that you'll encounter limits to the techniques before they do everything a person can do.
If there's a potential problem with machine learning that could sink the enthusiasm over time, I suspect it would come because machine learning applications are these black-boxes system which are the product of training with huge datasets and which use the very tuned-level of expertise of their creators (a common joke is talking of the "graduate school descent" algorithm, getting enough grad students to tune your app till it work). It may be that the deployers of these applications may find that when they have to train them again, in a year's time, that the geniuses have moved on to other things or that the geniuses now charge rates that look excessive for an application that works for just a year.
But that's just spinning possibilities. Currently things seem to be going great.
The other idea I have doted on is that perhaps universes are the manufacturing tool to create super AI by some parent universe.
> Taking the pessimistic view on science, what if we don't get there?
I have no interest in living forever but I really really wish I could be told what will happen or what did happen or what all is. I'm sure specialized intelligence will continue to improve but my gut says general intelligence is probably not in our lifetime (or is limited or capped from above mentioned pop-culture-probably-wrong reasons).
I ask because I wonder, if we did meet or make a creature which could learn everything, if it wouldn't say some variation on "99.9999% of everything there is to know is right there in your ecology and visible to the naked eye. Go look."
I also wonder if one of the first things the sentient AIs teach us is that yes, we are committing an egregious ongoing information Holocaust through habitat destruction.
And perhaps our creators are long dead but the simulation keeps going...
Given the enormous amount of press, tweets, blog posts, conferences, degree programs, seminars and interviews popping up it seems like there has to be something more than just hot air here. Still, the most outrageous predictions are hinged on breakthroughs in unsupervised learning happening. Taking the pessimistic view on science, what if we don't get there?