Re:Google T5 scores 88.9 on SuperGLUE Benchmark, approaching...
by briga on Thursday October 24, 2019 @10:30PM
(#21351285)
Limited headroom? Seems like they're assuming greater-than-human language ability is just impossible and will never be surpassed.
[ Parent
| on HN ]
Re:Google T5 scores 88.9 on SuperGLUE Benchmark, approaching...
by wyattpeak on Thursday October 24, 2019 @10:40PM
(#21351337)
I'd argue that greater-tham-human language ability is by definition useless.
Language is specifically a human communication tool, there's no value in surpassing the language skill that humans have, if indeed such a thing is even meaningful (what does it mean to be better than the best* French person at French?)
* By whatever language-related metric
Re:Google T5 scores 88.9 on SuperGLUE Benchmark, approaching...
by tremon on Friday October 25, 2019 @06:25AM
(#21353520)
I disagree, greater-than-human-average is not useless. There's a lot of room for misinterpretation in human language. We compensate for that by non-verbal communication (posture, expression) or by asking for clarification. On top of that, most places have local expressions or idioms that are not necessarily globally recognized.
So there's two ways in which a language automaton must be better than human: it cannot rely on non-verbal hints nor can it easily ask for clarification, and it must be able to interpret many different dialects and idioms correctly -- many more than an average human would need to.
Re:Google T5 scores 88.9 on SuperGLUE Benchmark, approaching...
by mannykannot on Thursday October 24, 2019 @10:51PM
(#21351391)
I do not think this result is that close to a greater-than-human language ability in general, and I do not think they are claiming it. I think the point is that, with scores on this test closely approaching average human scores, there is not much headroom for this particular test to drive, or measure, further progress.