The transformer can also be used at different levels of abstraction, e.g. to do interesting stuff with knowledge graphs. I think the transformer architecture is about to make things very, very interesting.

https://arxiv.org/pdf/1904.02342.pdf