I have the same feeling. Objects may be flying past our planet at many times the speed of light and we would be oblivious to them. I suspect the speed of light is the maximum speed an object can reach whilst still being detectable by conventional means. Once you surpass it things cease to be conventional.