Re:Recent Advances in Natural Language Processing
by nl on Sunday August 16, 2020 @09:10PM
(#24182817)
REALM is amazing, and Google's PEGASUS[1] taught itself to count between 2 and 5 (which is mindblowing).
Basically the NLP groups from Google and FB[2][3] are always worth watching.
[1] https://ai.googleblog.com/2020/06/pegasus-state-of-art-model...
[2] https://ai.facebook.com/blog/covost-v2-expanding-the-largest...
[3] https://ai.facebook.com/blog/introducing-a-new-large-scale-d...
Re:Recent Advances in Natural Language Processing
by MiroF on Sunday August 16, 2020 @09:18PM
(#24182869)
I think there's a pretty large gap between what people in academia NLP/industry labs find interesting and what the bitcoing/VC/whatever people on twitter seem to find interesting.
realm is definitely cool, and i agree that techniques like this are the way forward.
Lots of people are having fun playing with GPT-3 or AI Dungeon, myself included, but it seems like there is other interesting research going on like the REALM paper [1], [2]. What should I be reading? Why aren't people talking about REALM more? I'm no expert, but it seems like keeping the knowledge base outside the language model has a lot going for it?
[1] https://ai.googleblog.com/2020/08/realm-integrating-retrieva... [2] https://arxiv.org/abs/2002.08909