The way I think about it is by making an analogy with sleep: We perceive time to be fastest during sleep, i.e. when there is no sensory input. So the perception of time is actually linked to changes in sensory inputs. My hypothesis is that as we grow older, we become less and less sensitive to stimuli due to familiarity, so we perceive time to be faster.