I get what you're saying, but I don't think he was speaking explicitly about AGI. To me it didn't sound like they were leaving the applied problems he has worked on. Maybe I'm reading too much between the lines, but so far there haven't been any major successes in un-supervised learning models and he's suggesting here that rather than continue with the models that have had success up to this point, it would be more worth the energy to start again at a lower level, rethinking the problem, in order to achieve similar results unsupervised.

This may be where my limited knowledge is my handicap, but I didn't think you would need an AGI in order to achieve unsupervised learning. There would still be an input and an output as there is now -- but instead of massive datasets used in training the models could simple experience the problem in real time, adjusting and learning based upon new results. Of course unsupervised learning models could contribute greatly to the pursuit of AGI, but I didn't think AGI predicate unsupervised learning models (which is what I am taking from the article -- or more like article abstract -- here).

> Maybe I'm reading too much between the lines, but so far there haven't been any major successes in un-supervised learning models

Word2vec, a variant of which is Transform, Google's new machine translation framework which beats the pants off every other approach, to name one. The system it is replacing is also unsupervised.

Transformer is heavily reliant on supervised learning. Not sure where you're getting that from.

I was talking about word2vec.

I hadn't heard of that at all! Thanks. I'll check it out.