In science and engineering you almost never have a "major breakthrough" where some radical change happens over night. Progress is made by slow incremental improvements. Only from a historian's bird-eye view can you make statements like "the steam engine started the industrial revolution". Steam engines existed before James Watt. He just improved them a little to make them commercially viable.

AI in particular suffers from poor definitions. Nothing computers can do is "AI", only the things that are still impossible can be called "AI". Doing arithmetic was once thought to be a sign of intelligence. So was playing Chess, or Go, or recognizing faces. Now that computers can do this easily it is no longer considered difficult.

So while the recent progress with neural networks is likely somewhat overhyped you shouldn't discount the importance of making things commercially viable. In the 90s we had software that could do image recognition, true, but it wasn't accurate enough, general enough, fast enough to be useful in practice. This has changed over the last ten years to the point where image recognition has tons of commercial applications.

> Only from a historian's bird-eye view can you make statements like "the steam engine started the industrial revolution".

I'm always baffled by that example. The Industrial Revolution was already ongoing when the steam engines become practical.

Seems like it's unintentionally apt then ;)

It's revealing when you read older books pontificating about why computers can never be as smart as humans. They would say things like a computer will never be able to write poetry, prove a theorem or play chess like an expert. Now that computers have achieved all of that of course the goalposts have shifted.

So we have better, commercially available, versions of what existed decades ago. That's progress, but is it progress toward general AI?

Maybe. We don't know how to do general AI yet, so nobody can really tell.

Are DL-based reinforcement learners that observe the environment and make decisions a "mere pattern recognition" too? What is different about "true" intelligence then? What is major breakthrough? Self-consciousness?

Not self-consciousness, perhaps, but artificial instincts would be a next big thing. Imagine that your self-driving car is afraid of self-destruction and takes care of its own fitness by avoiding the holes on the tarmac.)

This, by the way, would be a major source of additional feedback for better training. Fear has been evolved for a purpose.

Wrong on so many levels. If AI has been well-researched in 90s, why it is DL that really pushes the boundary of several essential tasks, namely image/speech recognition, machine translation to near human performance?

I am curious about your definition of 'AI' that seems only obvious to the term 'Artificial'.

> namely image/speech recognition, machine translation to near human performance?

The key word here is recognition. Better, faster, more efficient classifiers.

The algorithms for training a better representation of the features, not just features itself is a big deal, but it is, again, supervised learning.

The adversary approach, which gives additional feedback for better training, is mere sophisticated supervised learning.

Take one of the best papers survey (which is an emergent new genre in hipster's blogs) and one will see that there is nothing fundamentally new, apart from sophistication and additional feedback.

While if you consider that human achieves intelligence through a lot of supervision(homework and exams) too, I won't go directly to disclaim only unsupervised learning can be hailed as the king.

In fact, there is a discussion about the definition of unsupervised learning, because of the emergence of techniques like word2vec, where it essentially supervised learning but construct all its tasks from the data itself without any labelling.

You do realize that main DL ideas have been developed in the 90s, or even earlier?

CNNs, RNNs, LSTMs, reinforcement learning, gradient descent, backpropagation - all that was used in the 90s, but given the limited computing power they had, people simply couldn't run large enough models on large enough data to get results.

Without development of technique like Dropout Batch normalization, residual connection, better optimizer like Adam, even with enough data and computational power, DL won't be nearly as powerful as it is today.

still there's good monet fooling vc around until they catch with the trick, that's why plenty startup sprinkle intelligence jargon on whatever they sell.

Yeah. We got money to build a medical instrument but sold it as AI to investors.