News for hackers. Stuff that matters.
Re:GPT Neo: open-source GPT model, with pretrained 1.3B & 2.... by f430 on Sunday March 21, 2021 @07:52PM (#26535239)
I see so that means this GPT Neo is 64 less powerful?
Re:GPT Neo: open-source GPT model, with pretrained 1.3B & 2.... by Voloskaya on Sunday March 21, 2021 @09:32PM (#26535875)
Accuracy and numbers of parameters don't scale linearly together. It varies widely depending on exactly what you are measuring accuracy on as well etc. But a very approximate rule of thumb would be to say that accuracy scales with the log of the parameter count (for the same architecture).