The reason is how far back their line diverged from ours. The nearer the point of divergence, the more reasonable it is to apply how things work for us.
The past to them is physically looking straight ahead, as that is what they can observe.
The future is conceptually and physically them looking over their shoulder behind them, as that is what they can’t see.
I might be mangling it somewhat! :-)
analogies develop further from there, such as events being passed at the moment moving very quickly, and large in one's field of view, and to some extent one js able to see around them and order their importance by size and speed with respect to one's own position. But they're moving fast and it's hard to take in all the detail at once, and being so close the more proximal ones obscure your view of the ones further away though the latter may actually be in fact larger or more significant. Only as they all recede into the distance of the past does their relative significance with respect to each other become more evident.
But there is no reason we should expect other species to communicate and think in the same way we do.