I am on a tangent here, but I don't think that Transformers will have very long term effects. It is highly resource intensive, and only a few company can take benefit for it.

I know HuggingFace has been a gamechanger providing API calls for big models, and providing very practically models like distilBERT, but still, Transformers don't seem practical to me.