Re:OpenGPT-2: We Replicated GPT-2 Because You Can Too
by godelski on Friday August 23, 2019 @01:03PM
(#20779716)
Well that is pretty dubious. That completely changes the metric of how much money and how difficult this costs. (And I definitely believe you because there's several other comments with that same number). Could be a typo? But I feel like that's something you'd say. Then again, these are masters students.
[ Parent
| on HN ]
Re:OpenGPT-2: We Replicated GPT-2 Because You Can Too
by p1esk on Friday August 23, 2019 @01:14PM
(#20779889)
It's $50k per training run. Their main contribution has been finding the optimal hyperparameters, not described in the OpenAI paper. Obviously you need more than one training run to to that.