News for hackers. Stuff that matters.

A comment on GPT Neo: open-source GPT model, with pretrained 1.3B & 2.7B weight models. Parent comment

Re:GPT Neo: open-source GPT model, with pretrained 1.3B & 2.... by MasterScrat on Sunday March 21, 2021 @09:20PM (#26535800)
GPT-NeoX, which is a model from the same group but using GPUs instead of TPUs, uses techniques from DeepSpeed:

https://github.com/EleutherAI/gpt-neox/

[ Parent | on HN ]

There are no comments on this comment yet.