DeepMind achieves SOTA image recognition with 8.7x faster training

Re:DeepMind achieves SOTA image recognition with 8.7x faster... by throwaway189262 on Sunday February 14, 2021 @02:41PM (#26135493)

Interesting, but in not sure you're completely right about XLNET. I heard it takes an absurd amount of resources to train. Even more than the BERT variations. And this is likely why there's not a ton of interest in it

[ Parent | on HN ]

Re:DeepMind achieves SOTA image recognition with 8.7x faster... by The_rationalist on Sunday February 14, 2021 @05:25PM (#26136854)

https://github.com/renatoviolin/xlnet XLnet running on very low end hardware (a single 8GB 2080 non ti) significantly outperform BERT large on e.g the reference question answering benchmarck: SQUAD 2 86% vs 81%

Nobody has even tried to create a spanXLnet (akin to spanBERT) How many years will be wasted before researchers get out of the BERT local minima? I'm a afraid it might last a decade

[ Parent | on HN ]