News for hackers. Stuff that matters.
Re:GPT-3 and Scaling Trends by ladberg on Wednesday June 03, 2020 @02:01AM (#23400321)
The author mentions that the shape of the curve from 117M parameters to 175B is interesting, but doesn't show it. Does anyone have the graph?
Re:GPT-3 and Scaling Trends by Tarean on Thursday June 04, 2020 @03:32AM (#23413714)
Presumably it's one of these graphs? https://i.imgur.com/r7xJfi1.jpg https://lh6.googleusercontent.com/VHmmdKYio39kz017ECfPwCxGPT...

There are a lot of graphs in the paper, though.

Re:GPT-3 and Scaling Trends by nostalgebraist on Thursday June 04, 2020 @12:02PM (#23418353)
I wasn't referring to one specific graph.

The GPT-3 paper (https://arxiv.org/pdf/2005.14165.pdf) has a very large number of graphs showing parameter count on the horizontal axis and some kind of prediction quality metric on the vertical axis. Most of the interesting ones are in Appendix H.

"The point isn’t the performance at 175B, but the shape of the curve as it passes from 117M to 175B" was referring to a general point about how to interpret any/all of those graphs, not a particular one of them.

Re:GPT-3 and Scaling Trends by yunusabd on Thursday June 04, 2020 @03:44AM (#23413811)
I think it's this one [1], there are also more detailed graphs in section 3 of the paper.

[1] https://imagehost.imageupload.net/2020/06/04/IMG_20200604_09...