I don't know that I agree with this first statement, but even if I do, everything is abstracted by import statements even outside ML. You say this is a huge amount of code abstracted, but it wouldn't be difficult to reimplement this in numpy and pandas directly without using tensorflow at all. The code would expand a bit, and you'd have to deal directly with backprop and calculating derivatives but it wouldn't expand things too much. But then you could make the same claim about numpy abstracting the linear algebra, and I could show you that I could extract that and do it without numpy but then it would be the python math library. It's turtles all the way down. My point is, your comment applies to everything.

Yup, I absolutely agree. Almost all big leaps in software engineering and applied computer science come from building a powerful and simple abstraction. Powerful and simple abstractions are surprisingly difficult to get right.

When people say "little code" what they mean an should say is "little customization of the tool".

There's also the issue of how hard it is to select the line of code (API call) that does the job (because the API surface is huge), which is nearly invisible attribute.

Mathematica is famous for being incredibly powerful with low custom coding, but also very hard to find the API call that does the precise thing you need.

We don't have good GPUs that is as fast until 580 (or to some extents, the first Titan). Previous generations only about 2 to 5x faster depending on what types of CPU you compare against.

IMHO, credit should always go to Alex Krizhevsky for the rapid spread of deep learning. He has shown us it was possible. Even without Tensorflow and PyTorch, we will be fine with Caffe, torch, mxnet or Julia.

I implemented neural networks before the advent of the good python frameworks. It sucked. And CNNs existed for decades before AlexNet. Honestly, the software and hardware engineers are the real heroes of the deep learning revolution.

By way of analogy, David Heinemeier Hansson didn't invent the webapp or even the MVC design pattern. But Ruby on Rails did change the way webapps were built, and enabled a bunch of stuff that wouldn't have been possible otherwise (or, at least, would've taken longer and been more expensive). Lots of websites were built because of Ruby on Rails that probably wouldn't have been built otherwise, even if people would've kept on doing the web thing regardless. We can say the same thing about lisps, about linux, and about a lot of other software infrastructure.

Any high schooler who's capable of learning python and can afford a gaming desktop can build and train a neural network. That's pretty amazing, and definitely wouldn't be the case without computation graph frameworks.

I've never worked for Google or with Jeff, and I'm not a huge fan of the ad tech industry, although I don't understand why either of those things should matter.