Google ‘BigBird’ Achieves SOTA Performance on Long-Context NLP Tasks

Re:Google ‘BigBird’ Achieves SOTA Performance on Long-Contex... by visarga on Monday August 03, 2020 @03:42PM (#24042396)

For an in-depth review you can see:

Yannic Kilcher - Big Bird: Transformers for Longer Sequences (Paper Explained)

https://www.youtube.com/watch?v=WVPE62Gk3EM

[ Parent | on HN ]

Re:Google ‘BigBird’ Achieves SOTA Performance on Long-Contex... by mdrabla on Monday August 03, 2020 @07:39PM (#24044804)

Thanks for the link - pretty good review. Totally agree with what he says about calling it SOTA. It's definitely a very interesting approach but I wouldn't put that much weight on whether it's achieved SOTA, because of the amount of compute they threw into it.

[ Parent | on HN ]