I think the issue is that these are small incremental improvements, whereas the current AI media discourse is making out like we are approaching the singularity at breakneck speed. The fact is that right now today there are relatively few domains where AI is really making a big impact in industry, and it just isn't clear how fast it's going to catch up in others (apart from self-driving cars which seem to be getting quite close).
AMT: Ok, so you have been working on neural networks for decades but it has only exploded in its application potential in the last few years, why is that?
Geoffrey Hinton: I think it’s mainly because of the amount of computation and the amount of data now around but it’s also partly because there have been some technical improvements in the algorithms. Particularly in the algorithms for doing unsupervised learning where you’re not told what the right answer is but the main thing is the computation and the amount of data.
The algorithms we had in the old days would have worked perfectly well if computers had been a million times faster and datasets had been a million times bigger but if we’d said that thirty years ago people would have just laughed.
As to the papers you mention- researchers publish papers and in them, they make claims. That's their job. The job of anyone who reads a paper is to take it in good faith but retain their critical and skeptical stance throughout. If you're not directly involved with ANN research, or implementing your own ANN systems for some commercial application, it's very hard to know whether a claim in a paper is useful to anyone besides the scientist who published it.
________________
[1] http://techjaw.com/2015/06/07/geoffrey-hinton-deep-learning-...
a) the algorithms are unimportant because all that we needed was computation
b) the algorithms were ahead of their time
I think that it is actually the latter that is true; in the same way that electricity was a precondition for computation. Babbage designed the difference engine and the analytical engine before they could realistically be implemented, this doesn't mean the design was unimportant.
"What we have seen lately, is that while systems can learn things they are not explicitly told, this is mostly in virtue of having more data, not more subtlety about the data. So, what seems to be AI, is really vast knowledge, combined with a sophisticated UX," one veteran told me.
Why be anonymous in such an article? You aren't leaking state secrets. I'd like this 'veteran' to explain to me why recent algorithmic advances like the DNC or recurrent entity networks are not "more subtlety about the data".
The last few years have seen major improvements in the underlying neural network algorithms alone, like the addition of 'neural memory', big advances in training of neural networks that have loops in them and so on. The major progress in fields like question answering, story comprehension and game playing aren't simply about using old techniques with more data. They do reflect significant theoretical advances too.
It's not a good article, just clickbait by Orlowski of the traditional sort. His final point about liability is so vague it could apply to industrial robots or autopilots too, somehow they manage to exist just fine.