https://github.com/renatoviolin/xlnet XLnet running on very low end hardware (a single 8GB 2080 non ti) significantly outperform BERT large on e.g the reference question answering benchmarck: SQUAD 2 86% vs 81%

Nobody has even tried to create a spanXLnet (akin to spanBERT) How many years will be wasted before researchers get out of the BERT local minima? I'm a afraid it might last a decade