Re:OpenAI Trains Language Model, Mass Hysteria Ensues
by SubiculumCode on Sunday February 17, 2019 @02:47PM
(#19186405)
To what extent is this not just finding text samples written in its training sample and regurgitating it near verbatim?? -Non ml guy
[ Parent
| on HN ]
Re:OpenAI Trains Language Model, Mass Hysteria Ensues
by HALtheWise on Sunday February 17, 2019 @03:23PM
(#19186644)
If you look at their paper, section 4 is entirely devoted to this question. They present compelling evidence that it is generating original content, the simplest of which is it's ability to write coherently about ridiculous things like talking unicorns that nobody has ever written about in the training set.
https://d4mucfpksywv.cloudfront.net/better-language-models/l...
Re:OpenAI Trains Language Model, Mass Hysteria Ensues
by arcticfox on Sunday February 17, 2019 @03:39PM
(#19186735)
The talking unicorns piece was shockingly good. That is at least as coherent of a news story than the average human could easily invent about it.
Reading that piece gives me the same weird feeling as watching AlphaStar playing through a StarCraft game.
Re:OpenAI Trains Language Model, Mass Hysteria Ensues
by applecrazy on Sunday February 17, 2019 @02:50PM
(#19186427)
You bring up a good point. Without seeing their code and training metrics, how do we know that this isn’t some extremely overfitted model?