This is a possible Fermi paradox answer, or at least factor. Maybe the vast majority of habitable worlds are larger than Earth. The inhabitants of those worlds might find space flight too hard or even impossible. They also might have less interest in it since they would make the natural anthropic assumption that worlds capable of supporting life are probably huge like theirs.