News for hackers. Stuff that matters.
Re:Facebook, Amazon, Google, IBM and Microsoft Create Partne... by pron on Thursday September 29, 2016 @12:14AM (#12603254)
> Deep learning is not a simple statistical clustering algorithm.

What is it, then?

In the 60s people used backpropagation to train neural networks. NN + BP is a very simple statistical clustering algorithm. I know that when I worked on neural networks in the 90s we still used backpropagation. Are they using something different now?

Re:Facebook, Amazon, Google, IBM and Microsoft Create Partne... by gormo2 on Thursday September 29, 2016 @12:49AM (#12603363)
>Are they using something different now?

Yes, why else do you think we've had such breakthroughs in the past six years?

Re:Facebook, Amazon, Google, IBM and Microsoft Create Partne... by pron on Thursday September 29, 2016 @04:59AM (#12604057)
No, I don't think so. AFAIK, deep learning is essentially the same 1960s algorithms[1] (possibly modified a bit) running on much larger networks. Most progress is due to better hardware (and ad hoc configurations, made possible by the larger networks afforded by better harder). Of course, SAT solvers, which have become extremely effective in recent years, are also still based on a 1960s algorithm[2], so use of an old algorithm doesn't imply lack of progress in effectiveness.

The two (NN and SAT solvers) share little theoretical progress (and certainly no theoretical breakthrough) in the past several decades, but SAT solvers aren't marketed as "AI" in spite of their seemingly magical abilities. I know that ML researchers usually cringe at the name AI and often try to disassociate themselves from the sci-fi term, but still, the marketing is extremely aggressive and misleading.

I realize that in every generation, marketers like associating the name "AI" with some particular class of algorithms, but it's important to understand that currently, assigning that name to this class of statistical clustering algorithms (regardless of their remarkable effectiveness in some tasks) is a stretch, just as it was when the term was assigned to other algorithms.

[1]: https://en.wikipedia.org/wiki/Backpropagation

[2]: https://en.wikipedia.org/wiki/Davis%E2%80%93Putnam_algorithm

Re:Facebook, Amazon, Google, IBM and Microsoft Create Partne... by nurettin on Thursday September 29, 2016 @04:07AM (#12603906)
They are mostly using the same Lenet-5 on modern hardware, just refined to converge better for some tasks.