Re:GPT Neo: open-source GPT model, with pretrained 1.3B & 2....
by Voloskaya on Sunday March 21, 2021 @09:32PM
(#26535875)
Accuracy and numbers of parameters don't scale linearly together. It varies widely depending on exactly what you are measuring accuracy on as well etc.
But a very approximate rule of thumb would be to say that accuracy scales with the log of the parameter count (for the same architecture).
[ Parent
| on HN ]
There are no comments on this comment yet.