AI used to be tiny - 20-50 people at CMU, Stanford, and MIT, and a few smaller groups elsewhere. All the '80s startups went bust. Now there are real applications that work and are widely deployed. Many classic problems, such as reading text and handwriting, have been solved and the solutions widely deployed commercially. Speech recognition is getting good and is deployed widely. Face recognition works. Automatic driving is working experimentally. There are now hundreds of thousands of people doing AI-type research.
Each new idea in AI has had a ceiling. Machine learning has a ceiling, but it's one high enough that the technology is good for something and generates enough revenue to finance its own R&D.
Machine learning is basically a form of high-dimensional multivariate optimization in possibly bumpy spaces. That's a hard problem, but considerable progress has been made, and the massive compute power necessary is now available. This is great. It's not everything, but it's real progress.
We're going to need another big idea to get beyond the ceiling of machine learning. No idea what that idea is.
I think that GAN's probably qualify - although you can see that emerging in the SAB series of conferences in the 90's if you read the papers.
On the other hand I do see a lot of small innovations that are enabling many people to create incremental improvements and applications. I feel that that the exploration of the field has been very weak and our overall knowledge is limited and not widely shared. Perhaps the improvements like MCMC search for bayesian reasoning, causality, counterfactuals, GPU's and TPU's and FPGA's and the access to very large data sets for training, forward training and so on will be the actual breakthrough.
It's natural to claim that's not generalizing since it's in such a specific domain as well as the fact that it had access to its score. But I think you have to consider that life itself starts in the exact same way. We're little more than a vast number of evolutions starting from very simple organisms who's only purpose was to seek out sustenance. They had an extremely simplified domain with limited input thanks to fewer senses. And they also received a score. They were 'informed' of their sustenance and if it fell below a certain level, that was game over which they were also informed of --- at least in some manner of speaking.
I think the ultimate issue is that it's rather disappointing how unexciting it all is. And so we constantly shift the goal posts. Going back not that many decades ago it was believed that a computer being able to defeat humans at chess would signal genuine intelligence. Of course we managed to achieve that, but the matter of how it was done being so unexciting and uninteresting led us to shift the goal posts.
The achievements we're regularly producing now a days dwarf anything those speaking of chess=intelligence times could have even imagined. Nonetheless we still now go, "Nah.. that's not REAL intelligence either." But I think we will, up to the day we do genuinely create generalized intelligence argue that it's not 'REAL' intelligence. And I imagine we'll be arguing that's not 'REAL' intelligence even afterwards. Because it won't be magical, or exciting. I don't think there's ever going to be a "we've truly done it" moment. It's just going to be clever ways of teaching increasingly sophisticated tricks leading up to the point that a creation becomes more effective at most of any task, including assimilating and producing new knowledge and expertise, than we are.
So much of what we mean when we say "this model generalizes well" seems like baked-in human assumptions about the data distributions that may not really have any basis in reality.
One way to look at what generalizability is would be like this: problem domains typically come with sets of axioms. Generalizability is the ability for a solution approach to work across different domains that don't have 100% axiom overlap. The wider the difference between axiom sets for a domain, the more difficult / impressive the generalization.
Solving two problems within the same domain, necessarily sharing a 100% overlapping axiom set, is not generalization at all.
The reason the axioms supporting the domain matter is that they (in part) guide which heuristics work, and how they should be applied. And that's the sort of generalizability that is missing from current work: some solutions can make some pre-programmed choices about applying different heuristics depending on the problem set (driverless cars are doing this now). This is the "bag of tricks" approach. But they don't typically morph in how they are applied, or the end that they're trying to accomplish when they're applied.
GAI is, to me, when machine is able to be given a problem and then, without prompting, decides which data to consume to learn how to solve that problem. It could be told to optimize an automotive design for a 5% efficiency increase without a loss of safety features and while keeping the performance the same - and then go out and figure out what data it needs to learn so that it can solve that task. It would assemble and process that data and then come up with the answer, which might just be that it is impossible with current tech and here is what is needed and this is how to do it.
That's rather verbose and I'm absolutely not the person who gets to define it. But, when someone says AI, that is how I think of it. More so when they say general AI.
It's just curiosity, I'm not expecting to enter the field of AI research. It's still fun to learn. Again, thanks.
When Data walks out of a lab, do you really think people will argue that he's not intelligent in the generalizable sense that humans are? Anyone who has watched ST:Next Generation agrees that he is.
> The achievements we're regularly producing now a days dwarf anything those speaking of chess=intelligence times could have even imagined.
I'm pretty sure the AI founders like Marvin Minsky could imagine Artificial Intelligence being generalized over various domains. I forget which one predicted the by the year 2000, an AI would be able to read all the world's books in an intelligent manner. McCarthy assigned a student to solving the robot vision problem over a summer back in the 60s.
Problem is that they underestimated the difficulties in getting a machine to perform tasks that are simple for us, while thinking that tasks like superhuman chess would be the difficult challenges. They had it backwards.
There is no problem domain that humans have been shown to hold an unshakable advantage. I used to be able to say Go but that is no longer even a little bit true. It could well just be time and data from here.
And it's not just 10-20 years of a blank slate being exposed to data. It's evolved brain structures that know how to learn those skills. That's why a young human child quickly surpasses the learning abilities of chimpanzee.
Yeah. I would like to see the same approach applied to NES games[0] with the capability of beating a game like Zelda or Ultima IV with no domain-specific knowledge. It's not going to happen. These games assume the player knows about human culture, values, virtues such as courage and triumph over the forces of evil. Symbols such as the sword and shield, the heart; they have deep meaning even to a person who has never played a video game before.
To build an AI that can play a game like that, without any specific knowledge of the game itself, would be to build something that likely reduces to general intelligence, though I hesitate to claim it forcefully.
[0] I am aware of this: http://www.cs.cmu.edu/~tom7/mario/
Much of the rest is logical inference.
You choose words in the order you choose them because as a child, your mind learned that, statistically speaking, certain words had certain meaning, and when used would engender certain results. All intelligent decisions follow that pattern. Training, comparison to expectations, memorize the difference, adjust behavior, rinse and repeat. Sound familiar?
Most of the intelligent mechanisms within the mind are employed through similar statistical inference, and when it's not done statistically (say, when you are inspired by some flash of 'insight' or when you dream things that have little obvious statistical grounding) there is still some reasoning within the mind that is allowing room for stochasticity to infer intelligence that is statistically calculated to be beneficial, when all other reasoning has failed. Tangentially, it might even be true that the irrational mind is simply one that uses stochastic reasoning more often than is deemed sane by our social norms.
Don't overstate intelligence like it's some amazing technology that we can't understand. Consciousness is more difficult to parlay into a black box model, but intelligence, reasoning, and 'wise' decision making is very much rooted in making statistically beneficial choices and deriving logic from the order we recognize via inductive and deductive reasoning.
If it sounds like I am oversimplifying, please understand that I hold the brain and the amazing parallel processing power it wields in high regard, and it's no mean feat for someone like Einstein, to use an obvious example of great intelligence, to intuit relativistic frameworks using only thought experiments, but his understanding (and everyone else's, for that matter) were built on many, many layers of statistical reasoning.
1. We're on the wrong track, and don't understand our own intellects at all.
2. Human intelligence is just a collection of these same hacks, possibly ensembled by some relatively thin meta-algorithm.
We should however think more expansively about intelligence than just the sort humans have. The goal doesn't need to be straight mimicry of humans
The thing about emergent behavior is that you just have to take enough entities and organize them the correct way, and the behavior appears, unexpectedly, and out of nowhere.
If it is really emergent, then nobody (what includes me and you) has no idea how far we are from a general intelligence at all.
In any case, I dispute your facts too. I don't think we have the ability to cross apply our heuristics. I can calculate fairly accurately a lot of mathematical problems when I'm riding my bike, but put me in a maths class and I'm stuffed. I don't have independent access to those hacks.
EDIT: I should also add that reinforcement learning more directly falsifies your claim.
Maybe our collection of hacks includes hack generation hacks (they cannot work reliably of course).
More so if you include reasoning illusions like people being more scared to catch a plane than drive a car or thinking that a lotto ticket is a good investment.
Human intelligence doesn't really meet intuitive definitions of human intelligence. But it does work well enough as long as you ignore all the times it doesn't.
http://karpathy.github.io/2014/09/02/what-i-learned-from-com...
Given that the author is writing a blog about AI, it's reasonable to assume he's above average in intelligence and knowledge, and therefore likely better than the average human at this task. But even if you don't make that assumption, object identification is within spitting distance of human level performance.
The same is true for many of these other tasks.
http://www.sciencemag.org/content/350/6266/1332.short
Machine translation is a joke.
Put any comment on this page through Google translate to another language and back to English and see what you get.
I did a small part of yours. Hardly human-level for just a small simple sentence.
> But even if you do not make this assumption, identifying the object involves spitting the distance from the performance of the human level.
Cognitive science, on the other hand, has arguably not made a lot of great progress. We don't really understand how we learn. We don't really understand how language works. Etc. But arguably a lot of this sort of thing needs to be part of "AI" to get beyond problems that are amenable to pattern recognition.
Ask them to state it - they memorize the text book definition.
Ask them to apply it to a specified problem - they scan the problem for values of variables and look up a formula list to find one that has those variables.
Ask them to explain why their answer is correct and they form a grammatically correct explanation made by plucking phrases from the problem description and linking them to the answer with "so" or "because".
It feels like they don't understand but they actually can get a long way (ie. not fail) like that - it's certainly human level understanding, even if it's not what the smartest of us are capable of.
Personally, I think understanding is a continuum from special case memorization at the bottom, up to being able to link with a lot of other concepts at the top. There's no bright line between "truly understands" and "illusion".
The big fundamental problems are all the same, and we don't appear to have much progress on them. We have figured out very clever ways of teaching extremely useful new tricks to computers. I don't mean to denigrate the value of that work, it just resembles regular programming, but by different means.
Whatever your definition of intelligence, there's little to no generalizability of what's happening right now.