"you have to draw the line somewhere"

For taxes and things, sure. For oppression of peoples? Probably. To keep undesirable countries citizens out? Sure. To have an excuse not to care about what goes on in other countries? Most definitely.

Travel and freedom of movement and work and choice in where you wind up living? Not really.