Sure there is, at least if you acknowledge the notion of human rights in the first place. (There's no "innate human right" to not be tortured.)
Without governments, there is no impedance to freedom of movement. This isn't merely theoretical: as governments have become stronger and their reach extended across the globe, the ability to move freely has correspondingly decreased.
This is a rude way of approaching a discussion. Don't make my assumptions about my knowledge.
A much better way of starting a discussion would be to state what you believe the definition of a human right is. There are in fact many different philosophical views on what human rights are, with many philosophers in fact arguing that their origin is in natural rights. [0]
> "Human rights" are something that didn't exist in the state of nature but were invented afterwards.
That's pretty much what is implied by my first line:
> Sure there is, at least if you acknowledge the notion of human rights in the first place. (There's no "innate human right" to not be tortured.)
[0] https://en.wikipedia.org/wiki/Philosophy_of_human_rights
No human rights are innate though, they're all man made.
Humans are individuals, individuals form into groups, and those groups when large and cohesive enough form into societies, civilizations, and now nation states. If those groups are to exist as a "group" they require these shared norms, values, culture to remain a viable group. If part of that culture is to exclude outsiders on the area of land on this earth that they control, then who is one individual or many individuals to say that this is invalid?
This presents a lot of opportunity for places like the United States where immigration and freedom of movement is more accepted - so you get more immigration and more growth. But that is something citizens of the US have decided, not external actors.