Re:Natural language benchmarks don’t measure AI models’ gene...
by tsimionescu on Monday August 24, 2020 @03:59AM
(#24258389)
By this token, the best way to learn Mandarin would be to start looking at text in Mandarin and trying to assign it meaning. While it theoretically can work, it is vastly more difficult than trying to relate it to a previously learned model. In fact, it is so difficult that we have never successfully understood a lost human language this way (though with the caveat that we don't have the huge corpus of text that GPT-3 was trained on in any of the lost languages we tried it on).
[ Parent
| on HN ]
Re:Natural language benchmarks don’t measure AI models’ gene...
by hackinthebochs on Monday August 24, 2020 @08:51AM
(#24260111)
No, I said the best way to learn to predict human text is to capture human knowledge. I did not say how best to capture human knowledge. Obviously doing gradient descent over a huge text corpus is massively inefficient for a human. Whether its the best way for a machine to learn it is another matter. After all, the amount of data that humans experience through their senses during formative years, as well as the "computational work" that went into evolving human brain architectures surely dwarfs GPT-3's training regime.