- Without needing to make it to the closest star, we have big problems here. If we solve those problems before we leave our solar system, we may be changed beyond recognition. We may not be biological any longer, for example, or at least not forcibly so, and traveling as solid matter may seem silly to our future descendants.
- We don't understand well enough the nature of reality. For all we know, our machines and organisms made of atoms and molecules may be, by far, more inefficient and wasteful than an equivalent process at some other layer or scale. Like somebody who discovers themselves living inside a match box in a forgotten attic, we may decide to move to the more spacious main floor of the castle.
- A variation of the above: maybe space-time itself is something we use inefficiently. It could be that a way to stop being troubled by the slow speed of light is by lowering our own "life" speed, increasing our volume to span entire solar systems, and decrease our density so much that your ancestors would confuse us with sparse interstellar matter. Or, at the opposite end, it could be that we find a way to move our entire future civilization to a cubic centimeter of space and a few microseconds that feel like eons.
Now let's take a much more reasonable speed like 10% the speed of light. Assuming the 100 protons per cubic meter figure above, each square metre of ship now only needs to dissipate 2.27 mW of energy. 10% the speed of light is enough time to reach Alpha Centauri within a single lifetime (42 years). And fast enough to visit every part of the galaxy in less than a million years. We could even imagine generational ships travelling at 1% the speed of light (now the energy dissipation demands are 2.25 μW per square metre of ship surface). That's still under 10 million years to colonise the entire galaxy.
If intelligent life is abundant in the galaxy then I don't think the speed of spaceships at least offers a fundamentally insurmountable technical challenge for that life to spread everywhere.
If the practical limits of rocket technology don't allow life to much beyond their own solar system then given the vastness of space that would be a good reason why we don't see any evidence of intergalactic civilisations or large feats of engineering. All other explanations for why the universe seems to lifeless seem to rely on elaborate hypotheticals like us being an early civilisation, us being extremely lucky/improbable in other ways, or that alien life is anti-social. But it always seemed to me that the best explanation is probably just that such things are not possible.
I mean there's a chance there's some new physics out there, but you'd think if there was star wars level tech out there (warp drives, etc) then something out there would have built one already and would rather quickly spread outwards...