Deep learning is not a simple statistical clustering algorithm.

Trial and error is a main way that people learn and it works.

> Deep learning is not a simple statistical clustering algorithm.

What is it, then?

In the 60s people used backpropagation to train neural networks. NN + BP is a very simple statistical clustering algorithm. I know that when I worked on neural networks in the 90s we still used backpropagation. Are they using something different now?

>Are they using something different now?

Yes, why else do you think we've had such breakthroughs in the past six years?

No, I don't think so. AFAIK, deep learning is essentially the same 1960s algorithms[1] (possibly modified a bit) running on much larger networks. Most progress is due to better hardware (and ad hoc configurations, made possible by the larger networks afforded by better harder). Of course, SAT solvers, which have become extremely effective in recent years, are also still based on a 1960s algorithm[2], so use of an old algorithm doesn't imply lack of progress in effectiveness.

The two (NN and SAT solvers) share little theoretical progress (and certainly no theoretical breakthrough) in the past several decades, but SAT solvers aren't marketed as "AI" in spite of their seemingly magical abilities. I know that ML researchers usually cringe at the name AI and often try to disassociate themselves from the sci-fi term, but still, the marketing is extremely aggressive and misleading.

I realize that in every generation, marketers like associating the name "AI" with some particular class of algorithms, but it's important to understand that currently, assigning that name to this class of statistical clustering algorithms (regardless of their remarkable effectiveness in some tasks) is a stretch, just as it was when the term was assigned to other algorithms.

[1]: https://en.wikipedia.org/wiki/Backpropagation

[2]: https://en.wikipedia.org/wiki/Davis%E2%80%93Putnam_algorithm

They are mostly using the same Lenet-5 on modern hardware, just refined to converge better for some tasks.

P.S.

I don't deny that it works, but it isn't AI (not that statistical clustering isn't possibly a foundation for AI -- we have no idea -- but the current state-of-the-art is a far cry from the sci-fi meaning of the term).

This stuff has been called AI for decades. Perhaps you are thinking of AGI.

AGI is a recent term coined precisely because the name AI has become confusing, which is precisely my point. It is also not widely known and used for the large part by people who believe in a sci-fi rapture-like event called the "technological singularity[1]", and so carries a bit of a cultish connotation. It doesn't do much to allay the confusion, though, as the term AI (even if understood to refer to something weaker than AGI) can be confused to be a component of AGI. I imagine that at some point, something called AI may eventually lead to AGI, but we have absolutely no idea whether what we call AI now is even on the right path, so the confusion remains.

It is true that some algorithm has been called AI for decades. It wasn't always this one or anything similar to it. Both Lisp and Prolog were thought to be AI languages at one point.

[1]: https://en.wikipedia.org/wiki/Technological_singularity

My point is that the word "AI" has been referring to "weak AI" for decades. Everything from video game AI, expert systems in the 80's, genetic algorithms, searching algorithms, etc, have been called AI. There is no point arguing about the word "AI" now.

The word AGI is pretty established. There are AGI conferences. I've heard it used by a wide variety of people, not just singularitarians. "Strong AI" is another common term.

I don't think there is much doubt at this point that neural networks are on the right path for AI. They are extremely general, have made remarkable progress in widely different AI domains, and are the closest AI approach to the human brain.

> I don't think there is much doubt at this point that neural networks are on the right path for AI.

Not only is there doubt, I don't think any NN researcher would even dare to suggest (based on scientific knowledge; not as a mere conjecture) that neural networks, and certainly current NN algorithms, have anything to do with AGI, which, at this point, is still a dream or a sci-fi concept.

> They are extremely general, have made remarkable progress in widely different AI domains

They work precisely where statistical clustering works, because that's what they are. Statistical clustering is extremely effective.

> and are the closest AI approach to the human brain.

We don't know that. It is possible, even likely, that statistical learning plays some low-level role in the brain. We know little beyond that, but it is pretty certain that neurons in the brain work very differently from neural networks. I don't think anyone imagines that backpropagation is used by the brain.

Shh. You're among blue monkeys. They stone green monkeys.