RETRO has external source of data but is something entirely different to what is discussed in the paper. In simplistic terms, in RETRO architecture you store embeddings in an external database and then use kNN search. It is not an end-to-end model capable of things like those with stack or tape memory (what OP asked about), retrieval models are only good for limited Q/A atm.