Space and time are one thing, which have to be thought about together. Your current thinking imagines that you're in a box, with x, y and z coordinates, and that time is a thing passing inside it. Instead, it'd be more accurate to talk about that you're in a frame of reference with x, y, z and a, and all are tied together. There's no sense in which you can talk about space and not also be talking about time, and vice versa. For a similar idea, a 3d volume is not a plane plus a z axis, where you can talk about moving through just the x and y axis without the z axis mattering. You can talk about a view of that, but it doesn't mean the z axis isn't relevant. Ask two planes not colliding whilst viewed from above how important a z axis is. (Maths jokes are the worst.)
The actual physics involved for discussing this gets absurdly complex very quickly, but this is about as simple as I can think how to explain it whilst still being in the bounds of accurate.