What are the limits of deep learning?

Re:What are the limits of deep learning? by evrydayhustling on Wednesday January 23, 2019 @01:09PM (#18980784)

Yet another piece that cherrypicks AI failures in order to make vague thesis that "deep learning is not enough". I can't decide if it makes the article more or less honest that the "Beyond Deep Learning" section is essentially a catalog of... recent developments in Deep Learning. The need to (selectively) pidgeonhole and then judge (on impossible to operationalize criteria) a fast-evolving field is purely rhetorical and, worse, aimed at folks who don't understand the way the field is progressing.

I've got nothing against raising platforms for alternative lines of AI research, especially including highlighting the ways they complement current deep learning But this article format is so transparent and tired.

[ Parent | on HN ]

Re:What are the limits of deep learning? by dmreedy on Wednesday January 23, 2019 @01:52PM (#18981341)

The criteria certainly are impossible to judge currently. Which makes your assertion that the field is 'progressing' strange.

I think there's a lot of merit to remembering the last time we got this excited about a given model for AI. And the time before that. And the time before that.

Go back further, to the time that we were all so excited that we could model the entire world with a little bit of beautiful logic. And the subsequent disappointment.

There is so much nonsense out there, so much hand-waving, mixed signals, false promises, downright naivety, and both malicious and completely inadvertent miscommunication. A little rigor in a field this excitable would be nice, I say.

[ Parent | on HN ]

Re:What are the limits of deep learning? by evrydayhustling on Wednesday January 23, 2019 @08:15PM (#18985132)

The progress is well defined. Both academic and industrial deep learning groups aggressively define challenge tasks in vision, NLP, relational learning and more, then document progress on them. There is even an unprecedented release of code and datasets to let others replicate and extend work.

Detractions like your "so much nonsense out there" pick straw man goalposts - whether it's AGI (whatever that really means) or "the craziest thing I heard from IBM marketing". The fact that some people are talking about crazy milestones doesn't mean that there aren't real ones being met.

[ Parent | on HN ]

Re:What are the limits of deep learning? by YeGoblynQueenne on Wednesday January 23, 2019 @04:26PM (#18983151)

Unfortunately, when rigor crawls in, excitement flies out. Rigorous machine learning would be really boring for everyone except its most dedicated followers. It's probably how Geoff Hinton and his students soldiered on with their research through alternating waves of lay hype and disappointment.

[ Parent | on HN ]

Re:What are the limits of deep learning? by RivieraKid on Wednesday January 23, 2019 @06:11PM (#18984171)

> Which makes your assertion that the field is 'progressing' strange.

Well, it is progressing. That assertion is correct.

[ Parent | on HN ]

Re:What are the limits of deep learning? by RivieraKid on Wednesday January 23, 2019 @06:26PM (#18984299)

Yeah, I started typing a comment trying to say something like that but was unable to put it into words.

I've only skimmed through the article but in general, these criticisms of DL are annoying, they rarely bring new or valuable information. It's usually a mix of disproving claims nobody is making (e.g. that DL is close to general AI) or stating something that's obvious.

[ Parent | on HN ]

Re:What are the limits of deep learning? by eli_gottlieb on Wednesday January 23, 2019 @01:28PM (#18981023)

If you actually read the papers they cite, you'll notice they bring in probabilistic programming as well as other alternatives to deep learning.

[ Parent | on HN ]

Re:What are the limits of deep learning? by evrydayhustling on Wednesday January 23, 2019 @01:37PM (#18981132)

I didn't say they didn't! Yes, they "bring in" probabilistic programming, graphical representations of symbolic structures, and more. But all of the citations 6-10 are from papers that integrate new approaches into a deep neural model. Most are from self- described deep learning teams, and all would be welcome at conferences and publications within the deep learning community.

Integrating with other inference paradigms has been a major driver in neural network research for 30y... so trying to cast progress like this as some sort of comeuppance that proves the insufficiency of the field is BS.

[ Parent | on HN ]

Re:What are the limits of deep learning? by eli_gottlieb on Wednesday January 23, 2019 @03:59PM (#18982884)

>But all of the citations 6-10 are from papers that integrate new approaches into a deep neural model.

Having read citation 8, "Building Machines that Learn and Think like People", a number of times over, I don't see where that paper includes a neural network architecture.

[ Parent | on HN ]

Re:What are the limits of deep learning? by evrydayhustling on Wednesday January 23, 2019 @08:59PM (#18985342)

Your correct - that one doesn't propose an architecture but suggests that others do so. It also is arguably from "outside" the main deep learning community, and in the context of the OP could be interpreted as corrective.

However, that is exactly not the paper's own stance, which is explicitly constructive to the field. And it is reciprocated! Check out all the NeurIPS papers citing and relating their work to the suggestions here: https://scholar.google.com/scholar?cites=8504723689348856287...

While I misspoke, my point remains that both this contribution and it's reception are part of a healthy community, not a case of "moving past" or exposing shortcomings of a field..

[ Parent | on HN ]

Re:What are the limits of deep learning? by eli_gottlieb on Thursday January 24, 2019 @11:40AM (#18990235)

>While I misspoke, my point remains that both this contribution and it's reception are part of a healthy community, not a case of "moving past" or exposing shortcomings of a field..

From within "the field", there isn't a single "deep learning community" counterposed to an "other stuff community". There's a broad machine learning community, in which deep learning is currently the most fashionable approach, and there are small factions of pro- and anti-"deep" partisans, who consider their approach the One True Approach.

Science is usually unkind to partisans, and so most people I've met don't really bother with that kind of partisanship. Asking, "Can deep learning get us all the way to AGI, yes or no?" is the wrong question. The proper questions are:

* What is deep learning good or bad at?

* What do we mean by AGI? (Similarly: data analytics, machine perception, statistical AI, etc?)

* Why should we target one goal (high-dimensional statistics, missing-data reconstruction, reinforcement learning, AGI) over another?

* What methods currently afford us the best contributions towards the scientific and engineering goals we're targeting?

[ Parent | on HN ]

Re:What are the limits of deep learning? by evrydayhustling on Friday January 25, 2019 @10:18AM (#18998642)

Sounds like we are on exactly the same page. Defining "the field" precisely is a rhetorical device. Defining and working on problems is what creates progress.

[ Parent | on HN ]

Re:What are the limits of deep learning? by sh33mp on Wednesday January 23, 2019 @04:21PM (#18983104)

The above commenter probably spoke a little too hastily. "Building Machines" is indeed not a paper about a neural network method, but a survey of problems that we should expect neural networks to do better on, but they currently aren't. That said, the paper isn't down on deep learning, but rather says "we need extra stuff, either additional methods or more inductive biases in our models".

[ Parent | on HN ]