Makes sense if you think about memory in the same way you think about time, as creations of the mind.

I read somewhere once about a story written in a book. The beginning and the end is there, everything superimposed together. For the characters in the book, there is no time.

Then the reader's mind/consciousness comes along and goes through the book, page-by-page. By this very act he creates memory and time. Maybe our minds are doing the same thing on an infinite, timeless substrate that Buddhism, Sufism, Zen etc call awareness.