News for hackers. Stuff that matters.

loxali

Re:Why GPT-3 Matters on Saturday July 25, 2020 @05:34PM (#23952602)
I think it's pretty clear that there is something more than memorising examples from the training set going on. Look at this: https://www.johnfaben.com/blog/gpt-3-arithmetic

In which GPT-3 answers the question "what is one hundred and five divided by three?" with "35.7". It also gave several other close-but-not-correct answers. It seems pretty unlikely these are all present in the training set, and surely can't all have been lifted verbatim.

I agree systematic testing is probably more useful, but find it really hard to believe this is all happening without any sort of model of arithmetic.