Re:OpenAI releases larger GPT-2 model
by czr on Saturday May 04, 2019 @08:59PM
(#19830423)
It seems to train fine on Colab without that flag, so that's a decent fallback for anyone running into memory issues on a local machine.
[ Parent
| on HN ]
If you did a fresh pull of that fork using a Colab notebook that does so, it will set the flag automatically if you specify the 345M model (I will follow that behavior in my own package).
If it didn't set the flag, it gets messy: https://twitter.com/minimaxir/status/1124742105421631488