News for hackers. Stuff that matters.
Re:GPT-3: A Disappointing Paper? by Eridrus on Saturday May 30, 2020 @12:01PM (#23362032)
I'm an engineer and this is a boring paper. They didn't beat the conclusively beat SOTA the way BERT/T5 beat basically everything. Sure, this is "unsupervised", but from an engineering perspective that's entirely uninteresting, so we don't even get actual scaling numbers. This paper was written to pretend it is more than it is, which is basically the hallmark of everything OpenAI does.
Re:GPT-3: A Disappointing Paper? by The_rationalist on Saturday May 30, 2020 @06:38PM (#23365450)
There is one model that consistently outperform BERT: it's XLnet but it's already one year old, innovation seems to slow down