News for hackers. Stuff that matters.
Re:OpenAI Trains Language Model, Mass Hysteria Ensues by applecrazy on Sunday February 17, 2019 @02:50PM (#19186427)
You bring up a good point. Without seeing their code and training metrics, how do we know that this isn’t some extremely overfitted model?
Re:OpenAI Trains Language Model, Mass Hysteria Ensues by vedant on Sunday February 17, 2019 @08:05PM (#19187973)
From the paper:

"All models still underfit WebText and held-out perplexity has as of yet improved given more training time."