News for hackers. Stuff that matters.

A comment on Are we in an AI Overhang?. Parent comment

Re:Are we in an AI Overhang? by ja3k on Monday July 27, 2020 @02:38PM (#23968029)
You don't have to wonder. In their paper: https://arxiv.org/abs/2005.14165 they state it has 0.7% accuracy on zero shot 5 digit addition problems and 9.3% accuracy on few shot 5 digit addition problems.
[ Parent | on HN ]
Re:Are we in an AI Overhang? by ralfd on Monday July 27, 2020 @03:53PM (#23968745)
By the way: Arithmetic accuracy is better if dollar sign and commas are added (financial data in the training set):

http://gptprompts.wikidot.com/logic:math

[ Parent | on HN ]
Re:Are we in an AI Overhang? by gwern on Monday July 27, 2020 @03:45PM (#23968679)
You do have to wonder, because as that section states, the BPEs may impede arithmetic, and as we've found using the API, if you use commas, the accuracy (zero and few-shot) goes way up.
[ Parent | on HN ]