News for hackers. Stuff that matters.
Re:Geoffrey Hinton spent 30 years on an idea many other scie... by gameswithgo on Saturday February 03, 2018 @02:46PM (#16299190)
Most other scientists dismissed neural networks? Is there some history I am unaware of as that doesn't seem true. Did the article want to push the idea of the lone rogue thinker a bit too much?
Re:Geoffrey Hinton spent 30 years on an idea many other scie... by meow1032 on Saturday February 03, 2018 @03:09PM (#16299326)
http://www.dataversity.net/brief-history-deep-learning/

Neural Networks weren't really thought of very highly until CNNs started winning image recognition competitions in the early 2010's.

I think most people had the feeling that they were interesting tools to learn how the brain worked, but too slow and opaque to be practical statistical tools. I've been following Hinton for a while (because of Hinton and Shallice 1991), and my understanding is that it was really hard for him to get funding especially when he was just starting out.

The fact that so much of the work from the mid-80's to 2000 came from just a few labs should tell you how hard it was to get funding for that kind of research.

Re:Geoffrey Hinton spent 30 years on an idea many other scie... by _delirium on Saturday February 03, 2018 @03:16PM (#16299376)
> The fact that so much of the work from the mid-80's to 2000 came from just a few labs should tell you how hard it was to get funding for that kind of research.

If you look through papers from that era I don't think you'll find that's true at all. You could fairly say that only a few of the prominent labs from the first neural-net boom lasted long enough to still be prominent labs now in the second neural-net boom (though even then there are several: Hinton, Bengio, Schmidhuber, LeCun, etc.). Since they're still around doing interviews and putting out new papers, understandably their work has a higher profile now than that of people who aren't in the field anymore. But there have been a ton of others over the years too, just many of them from the first wave have moved on or retired by now.

Especially in the '90s the field was hot and reasonably large (and it was pretty easy to get funding, too). If you look at e.g. the NIPS 1992 proceedings, it's definitely not just a handful of labs: https://papers.nips.cc/book/advances-in-neural-information-p...

Re:Geoffrey Hinton spent 30 years on an idea many other scie... by meow1032 on Saturday February 03, 2018 @07:29PM (#16300532)
>You could fairly say that only a few of the prominent labs from the first neural-net boom lasted long enough to still be prominent labs now in the second neural-net boom (though even then there are several: Hinton, Bengio, Schmidhuber, LeCun, etc.)

I agree, I don't mean to imply that Hinton did all of the work on NN's, but he certainly contributed a lot to the field, even as the popularity waxed and waned.

Even if you just click on a few of the names in your link, you can see most of those authors submitted ~1-6 papers. Hinton, Bengio, and Jordan have submitted >50 each. I wouldn't say that NN's were exactly thought of as a joke before 2011, but from the people I talked to about them, they weren't considered very promising as practical statistical systems.

Re:Geoffrey Hinton spent 30 years on an idea many other scie... by chriskanan on Saturday February 03, 2018 @05:26PM (#16300040)
Definitely seemed like that in the 2000s when I was doing my PhD. Yann LeCun even wrote an open letter to the computer vision community around 2012 complaining about them automatically rejecting papers that used neural networks. I've been told that letter led to him launching the conference ICLR. Only a couple years later he was a keynote speaker at CVPR, the big computer vision conference.
Re:Geoffrey Hinton spent 30 years on an idea many other scie... by dang on Saturday February 03, 2018 @02:53PM (#16299232)
Maybe, but let's not get hung up on the title—that tends to keep discussion shallow. We'll change "most" to "many" above.
Re:Geoffrey Hinton spent 30 years on an idea many other scie... by olympus on Saturday February 03, 2018 @06:35PM (#16300337)
I wouldn't wouldn't go so far as to say "dismissed," but they was definitely a period or two when they weren't as shiny.

When I did my M.S. reasearch in 2010 SVMs were far more popular than neural networks. Deep learning was just over the horizon and most people had given up on it being useful for learning real world problems. We had proven that NNs could learn anything given enough data and the right structure, but the hard part in real life was having enough data and finding the correct structure. So SVMs seemed more robust with available datasets, and most people used those. There were/are several other popular methods, like nearest neighbors and tree-based methods, and NNs just weren't as sexy as they seemed in the late 80's.

Re:Geoffrey Hinton spent 30 years on an idea many other scie... by gumby on Saturday February 03, 2018 @03:30PM (#16299455)
They are referring to Minsky and Papert's 1967 book "Perceptrons". Which said that a single layer NN can't be turing complete because it can't do XOR. The big 1990s breakthrough was having multilayer networks...which actually were also described in the Perceptrons book, though at the time computers weren't powerful enough to implement them.
Re:Geoffrey Hinton spent 30 years on an idea many other scie... by blt on Saturday February 03, 2018 @03:42PM (#16299523)
I don't understand how it took an entire book to show that a linear classifier can't learn the xor function.
Re:Geoffrey Hinton spent 30 years on an idea many other scie... by gumby on Saturday February 03, 2018 @06:21PM (#16300288)
There’s more to the book than my one-sentence summary. Believe Minsky and Papert have something extensive to say on a subject it’s probably not shallow. There’s an ok, if brief discussion about the book’s influence on NN research in its Wikipedia entry.

It’s not a super long book and an excellent, eye opening read even 50 years later.

Re:Geoffrey Hinton spent 30 years on an idea many other scie... by ehudla on Saturday February 03, 2018 @04:31PM (#16299781)
You should check out the book. It is really fun (especially the illustrations).
Re:Geoffrey Hinton spent 30 years on an idea many other scie... by anonytrary on Sunday February 04, 2018 @04:21AM (#16302025)
As time goes on, higher frequency computers will increase the number of AI tasks which are possible in a fixed period of time.

If you took half the apps built today and tried to run them on compatible hardware from 20 years ago, they would all flop.

Without the understanding of modern computing, one would have dismissed all such inventions as useless wastes of time.

Re:Geoffrey Hinton spent 30 years on an idea many other scie... by ZenoArrow on Saturday February 03, 2018 @02:50PM (#16299216)
I thought this too. Neural networks definitely seemed like a credible research topic in the 90s at the very least (which is when I first became aware of them).