It can be argued that there's an RNN-like process in the decoder stage but it's re-run every time based on a new context so it's not really "keeping up" in the same way as the human is.