Firstly, the universe's infinity is not a settled matter; current cosmology suggests it's vast but finite, but of course, there's no absolute answer to this (yet). I'd also look to the Drake Equation when considering this point, which illustrates the myriad factors and sheer number of uncertainties involved in the development of "advanced civilisations".
So, yeah: while advanced civilisations are certainly possible, they may not be as probable as you suggest, even in a universe brimming with stars and planets.
It's important to underscore just how big the universe is
There may be some more life out there. But even with lightspeed engines they are so far away as to not really exist in a meaningful sense.
With our limited technology w/r/t space, they are functionally non-existent
We currently have no evidence otherwise, and our best hopes (in my lifetime atleast) are finding ocean dwelling creatures on moons in the solar system that have no technology capable of reaching Earth.
It's just outrageous to think we are a unique possibility in the entire universe given how many stars and galaxies we have discovered.