News for hackers. Stuff that matters.
Re:Recent Advances in Natural Language Processing by gwern on Sunday August 16, 2020 @04:47PM (#24181171)
How ironic you claim that the paper overstates it, when you very carefully leave out every single qualifier about BPEs and how GPT-3's arithmetic improves massively when numbers are reformatted to avoid BPE problems. Pot, kettle.
Re:Recent Advances in Natural Language Processing by ximeng on Sunday August 16, 2020 @05:29PM (#24181495)
“Byte pair encoding”: more discussion at https://www.gwern.net/GPT-3#bpes