News for hackers. Stuff that matters.

A comment on GPT Neo: open-source GPT model, with pretrained 1.3B & 2.7B weight models. Parent comment

Re:GPT Neo: open-source GPT model, with pretrained 1.3B & 2.... by stellaathena on Sunday March 21, 2021 @11:39PM (#26536613)
It's because OpenAI sells them for profit. The "Ada" model is the same size as the larger of these two EleutherAI models.
[ Parent | on HN ]
Re:GPT Neo: open-source GPT model, with pretrained 1.3B & 2.... by minimaxir on Sunday March 21, 2021 @11:50PM (#26536662)
Huh, I was wondering what the size of the non-davinci models were; guess that make sense.

It's still telling that a "small" GPT-3 model can risk cannibalizing a larger model.

[ Parent | on HN ]
Re:GPT Neo: open-source GPT model, with pretrained 1.3B & 2.... by stellaathena on Monday March 22, 2021 @12:08AM (#26536771)
Ada is 2.7B, Babbage is 6.7B, Curie is 13.0B, and DaVinci is 175B. The new one they announced last month is in the 20-50B range I think, not totally sure though.
[ Parent | on HN ]