News for hackers. Stuff that matters.

A comment on What are the limits of deep learning?. Parent comment

Re:What are the limits of deep learning? by YeGoblynQueenne on Wednesday January 23, 2019 @05:24PM (#18983774)
I disagree. If I tell you that argoubr + vliugo - viyt = plqtv, do you now understand what any of those words mean? If I say that "gye" must always precede "turo", do you know what "gye turo" means, or why it doesn't make sense to say "turo" without "gye"? What kind of meaning is captured by the relations between these words that these rules I just described, define? Would it help if I gave you all the possible ways that every word in the language those words are drawn from are to be articulated together, without telling you what any of those wors mean? And how would I explain those words' meaning without recourse to words whose meaning you already know?

Also, that's what I mean by "structure" - the context in which tokens are found in text. I don't want to say "syntax" or "semantics" because it seems to me those are loaded terms that have very specific, er, meaning and I am not a linguist to be able to unpick it.

[ Parent | on HN ]
Re:What are the limits of deep learning? by yorwba on Wednesday January 23, 2019 @05:52PM (#18984033)
> Would it help if I gave you all the possible ways that every word in the language those words are drawn from are to be articulated together, without telling you what any of those wors mean? And how would I explain those words' meaning without recourse to words whose meaning you already know?

That's why I linked the two papers on unsupervised machine translation. Given enough text in an unknown language, it is already possible to turn those unknown words into words which humans understand.

[ Parent | on HN ]
Re:What are the limits of deep learning? by YeGoblynQueenne on Wednesday January 23, 2019 @06:28PM (#18984312)
But that's humans. Humans already understand language.

I don't disagree that humans can extract meaning from language models. But that only speaks to the ability of humans to extract meaning from even very badly formed language. Not to the ability of those models to represent meaning.

Edit: OK, I don't think I understand what you, er, mean. I think you're saying that language models essentially have a kind of understanding of language, and not just the ability to reproduce it without understanding it. Is that what you are saying?

[ Parent | on HN ]