However, I think you’re making a distinction without a difference going between classification accuracy and performance. That said, I am running on 1.5 hours less sleep than I need today — the only other important metric I can think of right now is the kind of skill which humans book-learn and which get implemented on a computer as an explicit and deliberate algorithm instead of being learned, and I simultaneously don’t count those as AI and think machines have beaten us for decades in that domain.
I do have one other question though: do you regard Alpha(Go|Zero|Star) as nothing more than classification?
I believe in AlphaZero etc the deep neural net component was used to identify moves with a high probability of leading to a winning board position. That's a good example of a problem we used to think would require inference or reasonging but that can, after all, be solved by classification.
Although that's not to say that the same problems can't be solved by inference, plus powerful computation. That remains to be seen.
Edit: Maybe I shouldn't have called it "dumb" classification; that sounds dismissive. There's no doubt that neural net classifiers are impressive in what they do. I mean it to say that they have no understading of their domain, or ability to reason etc.
I don't think we can say for sure that there is more, though. If and when we finally understand how it all works, I wouldn't be completely surprised to find that it is just many layers of pattern matching and associated feedback loops.
We just don't know enough yet.
AI systems are better than humans at making decisions. AI systems can scale to millions of decisions per second and beyond. AI systems don't need to spend 3/4 or their time sleeping or relaxing. AI systems don't need 20 years of training to become experts in their fields, they can be replicated almost instantly.
While the scope of decisions the AIs can do is somewhat limited at the moment, the trend is clear: in the not so distant future AIs will make all economically relevant decisions.
I don't doubt that, just doubt that they will make unanimously good decisions, as expected of a superhuman system. AI systems are yet to be "intelligent" which makes half of their name a PR gag.
Pattern recognition is simply not equivalent to intelligence. It is certainly a pillar of intelligence, however.
> AI systems are better than humans at making decisions
Are they? Dermatology, transportation, face recognition, NLP... in all these fields we are yet to reach human-like performance outside of ultra-specific tasks. There is a difference between recognizing a human face every time, even if obscured by sunglasses or half hidden behind a pint of beer, and recognizing 100.000 human faces per second with an error rate of 8% because of shadows.
Decent performance can be reached when you train AI to do one hyper-specific thing under just the right circumstances with mountains of data which need to be prepared just right. Otherwise you end up with bias and other issues. I wonder how AlphaGo would have performed if it was required to suddenly switch to Mahjong or cooking but I suppose that wasn't its purpose.
Please also mail me a link for systems that prepare training data to automatically and reliably prevent bias and other major issues that lead to malperforming systems.
Demonstrations optimized for PR, AI companies which employ humans as AI-pretenders, and catastrophically failing systems are the reality of AI today. I don't doubt that we will improve but superhuman AI requires more than mere linear improvement from where we are right now.
What is really the case is that some AI systems (in particular, deep neural net classifiers) have superhuman accuracy and that again only in classification tasks.
Edit: I'm not being contrary for the sake of it. I think there is a very useful insight to draw from the success of deep neural network classifiers: that it is possible to perform classification without any kind of understanding of the objects, or their classes, at all. In the past, AI researchers operated under the assumption that reasoning and inference would be required to perform this task, but we know now that such abilities are not necessary if the task is only to classify objects. It's also clear that in some cases dumb classification can replace reasoning- as long as a reasoning problem can be reduced to dumb classification.
However- humans clearly have the ability to reason and draw inferences (regardless of how often we do that successfully). There must be some explanation for this. Why do we have an intelligence that goes beyond simple object recognition? What is the point of having a broad intelligence? It must mean that there is more to intelligence than tasks that can be reduced to classification. So reducing "performance" to "accuracy" (of classification) risks fudging an important difference between the "superhuman" abilities of machines and human abilities.
In the end, the question is what are humans good at and why can't machines do the same things, even though they can beat us roundly in other tasks? What is the difference between tasks that are easy for machines and tasks that are easy for humans?