News for hackers. Stuff that matters.
Re:Google’s Neural Machine Translation System by gradys on Tuesday September 27, 2016 @06:35PM (#12593788)
I'd extend hypothesis #2. We might not have a sufficiently high-quality theory, but we might also not know how to integrate the benefits of the theory we have with the benefits of data-driven approaches to NLP like those described in this paper.

It's clear to me that data-driven techniques are an essential part of the answer to the problem of language understanding. It's infeasible to design a complete set of rules for understanding natural language with all its exceptions and fuzzy relationships. I'd point to Wittgenstein for a stronger philosophical argument on this.

That's not to say that theory has no place in language understanding, but it's not immediately obvious how to combine rule-oriented theory with data-driven ML.

There has been work in this area though, and actually several important, state-of-the-art-moving contributions have been inspired by linguistics. My favorite are recursive neural nets, which benefit from the linguistic insight that language has recursive structure. This paper[1], for example, describes a system that combines a dependency parser with a recurrent neural net (gross oversimplification) to yield a state-of-the-art sentence understanding model.

[1] - http://arxiv.org/abs/1603.06021

Re:Google’s Neural Machine Translation System by j-pb on Wednesday September 28, 2016 @06:45AM (#12596682)
> which benefit from the linguistic insight that language has recursive structure.

Really? You needed an entire _field_, and not just, and by just I mean, in an, but not too, extended, sense, really, trivial observation, to get to that conclusion?

Re:Google’s Neural Machine Translation System by MarkMc2412 on Wednesday September 28, 2016 @02:51AM (#12595884)
Do you have any further detail or reference to Wittgenstein here? Curious to learn more
Re:Google’s Neural Machine Translation System by thegreatestape on Wednesday September 28, 2016 @03:49AM (#12596066)
Not OP, but Wittgenstein covers this in his Philosophical Investigations. In the Tractatus, he tried to create an all-encompassing system for language, but in PI he more or less says it's impossible. It's an amazing read in any case, it has totally changed the way I approach philosophy.
Re:Google’s Neural Machine Translation System by MarkMc2412 on Wednesday September 28, 2016 @03:12PM (#12600400)
Ok, thanks!