“Deep Learning has outlived its usefulness as a buzz-phrase”

Re:“Deep Learning has outlived its usefulness as a buzz-phrase” by cs702 on Friday January 05, 2018 @10:40PM (#16083711)

I wish we could come up with a catchier name, but I LOVE the idea of calling this programming, because that is precisely what we do when we compose deep neural nets.

For example, here's how you compose a neural net consisting of two "dense" layers (linear transformations), using Keras's functional API, and then apply these two layers to some tensor x to obtain a tensor y:

  f = Dense(n)
  g = Dense(n)

  y = f(g(x))

This looks, smells, and tastes like programming (in this case with a strong functional flavor), doesn't it?

Imagine how interesting things will get once we have nice facilities for composing large, complex applications made up of lots of components and subcomponents that are differentiable, both independently and end-to-end.

Andrej Karpathy has a great post about this: https://medium.com/@karpathy/software-2-0-a64152b37c35

[ Parent | on HN ]

Re:“Deep Learning has outlived its usefulness as a buzz-phrase” by sitkack on Saturday January 06, 2018 @02:36AM (#16084305)

NN are _just_ transfer functions. Look up tables. Or really dense maps. So f(g(x)) make total sense. But I dont think these are the interesting combinations. I think giving one NN the training experience of another, plus the feedback on "correct inference" will be when on NN trains its replacement.

[ Parent | on HN ]

Re:“Deep Learning has outlived its usefulness as a buzz-phrase” by cs702 on Saturday January 06, 2018 @01:34PM (#16086711)

Yes, of course. f(g(x)) was the simplest possible example I could come up with to illustrate the point :-)

[ Parent | on HN ]

Re:“Deep Learning has outlived its usefulness as a buzz-phrase” by bmc7505 on Friday January 05, 2018 @10:47PM (#16083732)

  I hate the name, but LOVE the idea of calling this programming...

What would you call it instead?

[ Parent | on HN ]

Re:“Deep Learning has outlived its usefulness as a buzz-phrase” by letlambda on Saturday January 06, 2018 @01:59AM (#16084216)

∇programming

[ Parent | on HN ]

Re:“Deep Learning has outlived its usefulness as a buzz-phrase” by cs702 on Friday January 05, 2018 @11:00PM (#16083774)

I changed "hate the name" to "wish we could come up with a catchier name," which better reflects what I meant to write.

[ Parent | on HN ]

Re:“Deep Learning has outlived its usefulness as a buzz-phrase” by seanmcdirmid on Friday January 05, 2018 @11:03PM (#16083782)

LeCun specifically calls out imperative programming, not just typical data flow methods.

[ Parent | on HN ]

Re:“Deep Learning has outlived its usefulness as a buzz-phrase” by cs702 on Friday January 05, 2018 @11:09PM (#16083805)

You're right. I softened the reference to functional programming.

[ Parent | on HN ]