I used to think that the feud over the origins of backprop was an annoying one-off, but it is not. The field has had many examples of lazy citation patterns, including people not actually reading what they cite, and/or ignoring corrective papers like this one.
The example that bothered me most was the "Gaussian processes" boomlet. It played for a couple of years, but it was basically linear prediction theory redone by people who did not adequately cite prior work nor convey its limitations.
What I'm curious about is how such hype cycles affect our understanding of whether this shit actually works.
As far as I can tell, the "great advance" attributed to deep learning is doing benchmarks better than other methods. Which seems to have the result of doing the same old machine learning tasks more accurately than previously.
The main thing we hear about is more "edgy" stuff like learning video game by oneself, describing pictures with phrases or giving answers to "philosophy". But since these applications are easy to cobble together in a half-assed fashion, is there really more here than incremental progress?
I wouldn't know one way or the other, so I'd like to find out.
We are inventing new "real world" benchmarks to try and counteract this (MS COCO, dialog datasets, the big flickr datasets, translation generally), but many approaches from the 90s were clearly right mathematically (as Jurgen says) and just needed more data fuel. So it is obvious to go back and find interesting ideas that didn't get their due as long as proper attribution is given. Profs also have their pet projects that didn't quite pan out, and often want to breathe new life into a cool idea.
These things are only easy to half-assed cobble together in hindsight AND/OR if you have expertise - having the knowledge and know how to input conditional information, interpreting deep networks as modeling joint probability distributions, etc. is just as much algorithmic design as any other task in graphical modeling, statistics etc. Slapping a big deep convnet (or feedforward net) on new datasets IS easy, and usually not interesting scientifically, but also doesn't get published and is reserved for the blogosphere or bad ArXiV papers.
Incremental progress is 0.5% performance gains in major benchmarks like ImageNet etc. - company PR (and university PR as well) will crow about this but no one in academia really cares unless it is accompanied by interesting scientific ideas or fundamental questions being answered.
And to add onto this, another academic is getting his jimmies rustled because he didn't get the money his former PhD students did??
Heavens to Betsy!
---
Everyone self promotes and does things in their best interests. There is no clear divide between academic and industrial interests. Maybe in something more pure where truths are evident (e.g. pure math). But not something like machine learning, where your success and funding depends on armies of grad students fine-tuning stuff like the number of "hidden units" in some over-complicated model whose ultimate goal is to over fit a training set and cause more hype, etc.
Nothing wrong with this though, progress happens continually, just not linearly: https://en.wikipedia.org/wiki/Hype_cycle