When I am on a longer vacation and I do not use a certain password, I struggle remembering it.

If I were chucked into a prison and let out after several years with no computer use in between, I would likely forget all my passwords in the meantime. No brain damage needed, just disuse.