Impressive! With this work they managed to increase quality for most language pairs while halving runtime computational cost and decreasing training time. Since it's a gerneric component, I can see it getting used on all sorts of problems with lots of data.

Especially interesting given that most Kaggle competitions are won with mixture of experts models.