2) Read the article that you are referencing please. What you are implying is not the thesis of that article nor is it what Geoffrey Hinton is saying.

What Hinton is saying is, we should throw out deep learning. What he is saying is that the current approaches to AI are fundamentally broken and aren't going to result in artificial general intelligence.

Backpropagation is not just used in supervised learning, it is also used in unsupervised learning. I happen to agree with Hinton, in that there is too much hype around the current state and successes of AI, which has mainly been in "narrow AI".

AI is a term that gets thrown around a lot these days. But there is a big difference between technology that automates the tedious tasks of daily life, and artificial general intelligence.

In the world of deep learning, data is king.

>After just three days of self-play training, AlphaGo Zero emphatically defeated the previously published version of AlphaGo - which had itself defeated 18-time world champion Lee Sedol - by 100 games to 0. After 40 days of self training, AlphaGo Zero became even stronger, outperforming the version of AlphaGo known as “Master”, which has defeated the world's best players and world number one Ke Jie.

1) AlphaGo Zero was indeed trained in the way I mention.

2) As directly quoted from the article, Hinton believes that a better way of learning doesn't require all that labeled data. If such a method is invented, as is required to push AI forward, big corporations would not have a data advantage, which is my original point.

https://storage.googleapis.com/deepmind-media/alphago/AlphaG...

AlphaGo scratch: https://www.nature.com/articles/nature24270.epdf

I was specifically talking about AlphaGo, not AlphaGo scratch. Also, if you read the paper about AlphaGo scratch, the key innovation driving the self-learning is the use of MCTS as a policy improver, which couldn't have feasibly been done without AlphaGo and the supervised learning.

And I think Hinton is saying that we need fundamental breakthroughs for AI, and I don't think he is in favor of "traditional" modern neural network architectures. Anything that requires SGD, he doesn't like.

AlphaGo Zero is the product of incremental improvements made by the same group of computer Go experts that built the original system. _They_ learned from each experiment and incorporated that knowledge in the next version. It was not obvious _a priori_ that the AlphaGo Zero architecture or training method would succeed; if it had been, then the AlphaGo team would not have built the earlier versions. And while it runs on "only" 4 TPUs for inference, those TPUs are each about 30x as powerful as the computers that the original AlphaGo ran on, so it's more like a reduction from ~180 GPUs to 120 equivalent GPUs.

I am not disputing that having more financial resources would help anyone hire talent and build awesome infrastructure, and certainly the latest way of training AlphaGo Zero was aided by earlier experiments that relied on labeled data and extensive computational effort. However, by no means do I think big corps have a lock on these kinds of advancements. There will always be great people who would rather go the startup route, and both algorithmic and hardware advancements are drastically reducing the operational cost of training AI systems. Thus, when it comes to AI, I think very small teams will be able to get very far with the right approach.