It's a huge amount of code, hidden behind the tensorflow import statements. It's common to credit GPUs for the rapid spread of deep learning, but good GPUs were available for quite a few years before deep learning really took off. As someone who wrote * a lot* of OpenCL code, including my own python wrappers, I'm fairly certain this code would be thousands of lines without a computation graph framework library. These frameworks are really amazing pieces of software engineering and deserve some non-trivial fraction of the credit for the rise of deep learning.

If you want to know what the next hot thing in software engineering will be, just pay attention to whatever Jeff Dean is doing.

I don't know that I agree with this first statement, but even if I do, everything is abstracted by import statements even outside ML. You say this is a huge amount of code abstracted, but it wouldn't be difficult to reimplement this in numpy and pandas directly without using tensorflow at all. The code would expand a bit, and you'd have to deal directly with backprop and calculating derivatives but it wouldn't expand things too much. But then you could make the same claim about numpy abstracting the linear algebra, and I could show you that I could extract that and do it without numpy but then it would be the python math library. It's turtles all the way down. My point is, your comment applies to everything.

Yup, I absolutely agree. Almost all big leaps in software engineering and applied computer science come from building a powerful and simple abstraction. Powerful and simple abstractions are surprisingly difficult to get right.

When people say "little code" what they mean an should say is "little customization of the tool".

There's also the issue of how hard it is to select the line of code (API call) that does the job (because the API surface is huge), which is nearly invisible attribute.

Mathematica is famous for being incredibly powerful with low custom coding, but also very hard to find the API call that does the precise thing you need.

We don't have good GPUs that is as fast until 580 (or to some extents, the first Titan). Previous generations only about 2 to 5x faster depending on what types of CPU you compare against.

IMHO, credit should always go to Alex Krizhevsky for the rapid spread of deep learning. He has shown us it was possible. Even without Tensorflow and PyTorch, we will be fine with Caffe, torch, mxnet or Julia.

I implemented neural networks before the advent of the good python frameworks. It sucked. And CNNs existed for decades before AlexNet. Honestly, the software and hardware engineers are the real heroes of the deep learning revolution.

By way of analogy, David Heinemeier Hansson didn't invent the webapp or even the MVC design pattern. But Ruby on Rails did change the way webapps were built, and enabled a bunch of stuff that wouldn't have been possible otherwise (or, at least, would've taken longer and been more expensive). Lots of websites were built because of Ruby on Rails that probably wouldn't have been built otherwise, even if people would've kept on doing the web thing regardless. We can say the same thing about lisps, about linux, and about a lot of other software infrastructure.

Any high schooler who's capable of learning python and can afford a gaming desktop can build and train a neural network. That's pretty amazing, and definitely wouldn't be the case without computation graph frameworks.

I've never worked for Google or with Jeff, and I'm not a huge fan of the ad tech industry, although I don't understand why either of those things should matter.

"How can I train the model using my own raw data?

Generally, you just need to subtract the right black level and pack the data in the same way of Sony/Fuji data. If using rawpy, you need to read the black level instead of using 512 in the provided code. The data range may also differ if it is not 14 bits. You need to normalize it to [0,1] for the network input."

The Sony and Fuji training code looks mostly the same - they haven't bothered to pull out common code and re-use.

This could actually be shortened, maybe simplified, significantly. For example there is a lot of redundancy in the layers and that could be pulled out into a function per block. This is what I often do with deep networks as it helps avoid errors in code and shortens everything significantly, at the potential expense of being able to grok it initially as quickly.

But, many DNN concepts (and ML concepts themselves) can be described with a few lines of pseudocode. CNNs, RNNs, etc. can all be described in a few lines.

It's really quite amazing, most of the work goes into first creating the net work from theory, then training and tuning it until you get good results.

Do you have an iPhone 6s? https://youtu.be/qWKUFK7MWvg?t=85