The quoted remark assumes the universe was a state-machine. Which is a tall order. On the other hand, it's just a reformulation of the "energy is never lost" hypothesis (axiom), or "entropy always increases". But that's really just pop-sci. As was pointed out up-thread, it's rather describing the human way of thinking. Which to me also implies that we like to exaggerate a little: "Of course I could figure out where you were last month, in principle, it's just ... I don't have time for that right now!"

Edited: I always mix up increase and decrease of Entropy.