Another point is that once a civilization becomes advanced enough to be capable of intergalactic travel, maybe its resource needs change. The earth is great for us, but who says aliens have any need to visit here.