The speed of light is a limiting factor, but most models I have seen of galactic civilizations take that into account, yet they project that interstellar expansion of a hypothetical civilization is possible.

There are other factors that are more likely to explain why we are apparently alone: civilizations might not last long enough that they would encounter us before dying out, or that they have not yet emerged. IOW we might be alone in time more than we are alone in space.