I suspect when people are asked what they regret and they say, I wish I had spent more time with my family, what they really mean is, I wish I had more time to spend with my family.

Humans are not actually very good at remembering their past selves. Memories are reconstructed on the fly, we 'remember' by projecting our present selves into past situations.