News for hackers. Stuff that matters.
Re:GPT Neo: open-source GPT model, with pretrained 1.3B & 2.... by Voloskaya on Sunday March 21, 2021 @09:32PM (#26535875)
Accuracy and numbers of parameters don't scale linearly together. It varies widely depending on exactly what you are measuring accuracy on as well etc. But a very approximate rule of thumb would be to say that accuracy scales with the log of the parameter count (for the same architecture).

There are no comments on this comment yet.