News for hackers. Stuff that matters.

A comment on GPT-2: 6-Month Follow-Up. Parent comment

Re:GPT-2: 6-Month Follow-Up by gwern on Tuesday August 20, 2019 @02:39PM (#20750428)
They haven't released such models, though, and I don't know if it would be drop-in compatible with the OA GPT-2-774M checkpoint (they're training their own GPT-2s using their own webtext corpus).
[ Parent | on HN ]
Re:GPT-2: 6-Month Follow-Up by JonathanFly on Tuesday August 20, 2019 @04:03PM (#20751433)
I haven't look into at all myself, but he also said:

>We do provide training code that should work out of the box for gpt2 117M/345M

https://twitter.com/TheRealRPuri/status/1161319745259393024

[ Parent | on HN ]
Re:GPT-2: 6-Month Follow-Up by p1esk on Tuesday August 20, 2019 @04:18PM (#20751613)
It would take forever (or $$$) to train even 117M model from scratch.
[ Parent | on HN ]
Re:GPT-2: 6-Month Follow-Up by JonathanFly on Tuesday August 20, 2019 @05:02PM (#20752107)
I read that meaning you can start with the actual pre-trained GPT-2 models but I never got an answer when I specifically asked if that was the case.
[ Parent | on HN ]