It depends on the problem domain. Transformers are useful for NLP (language modeling, machine translation), but RNNs (and CNNs) are still being used in speech-to-text. When the input/output relationships are monotonic, transformers are probably too general.