News for hackers. Stuff that matters.

A comment on OpenGPT-2: We Replicated GPT-2 Because You Can Too. Parent comment

Re:OpenGPT-2: We Replicated GPT-2 Because You Can Too by the8472 on Friday August 23, 2019 @03:37AM (#20775282)
You're off by an order of magnitude and omit the caveats to that cost estimation. From the article:

> The cost of training the model from scratch using our code is about $50k. It’s important to note this figure is the estimated value of the cloud compute, and does not reflect the much smaller intrinsic costs involved

[ Parent | on HN ]
Re:OpenGPT-2: We Replicated GPT-2 Because You Can Too by p1esk on Friday August 23, 2019 @09:20AM (#20777232)
They edited the article after I left a comment there. The original text stated they spent $500k to run all the hyperparameter search experiments to replicate OpenAI results. Only after they did all that work you can run their code for $50k.
[ Parent | on HN ]