News for hackers. Stuff that matters.

A comment on OpenGPT-2: We Replicated GPT-2 Because You Can Too. Parent comment

Re:OpenGPT-2: We Replicated GPT-2 Because You Can Too by p1esk on Friday August 23, 2019 @12:55PM (#20779604)
It was in the article last night, they deleted it for some reason after my comment.

One of the authors has a peer reviewed NLP publication [1], the other has several publications in computer vision. I don’t know how they got research credits from Google.

[1] https://arxiv.org/abs/1905.13153

[ Parent | on HN ]
Re:OpenGPT-2: We Replicated GPT-2 Because You Can Too by godelski on Friday August 23, 2019 @01:03PM (#20779716)
Well that is pretty dubious. That completely changes the metric of how much money and how difficult this costs. (And I definitely believe you because there's several other comments with that same number). Could be a typo? But I feel like that's something you'd say. Then again, these are masters students.
[ Parent | on HN ]
Re:OpenGPT-2: We Replicated GPT-2 Because You Can Too by p1esk on Friday August 23, 2019 @01:14PM (#20779889)
It's $50k per training run. Their main contribution has been finding the optimal hyperparameters, not described in the OpenAI paper. Obviously you need more than one training run to to that.
[ Parent | on HN ]