I think that the older you get, the less new milestones/events you have (that impress you and remember in detail). This results in having less moments to refer to when looking back in time, (skipping uneventful timespans). This makes things seem closer to the current time than they really are as everything in between is empty noise.