Paupers from London didn't get on ships to the new world - not without selling themselves into indentured servitude.
When the economic barriers to migration fell to the point that subsistence farmers could travel between continents, we were always going to see either an equalization of living standards or the rise of political barriers.
Even before the adoption of visas, the traveling experience for a first class passenger on a ship was very different from third class getting routed via Ellis Island in the US.
Part of the reason for Britains diverse regional dialects is due to this.
[1] https://broadstreet.blog/2021/05/07/serfdom-the-state-and-th...
That being said, if you settled somewhere in some large empty land, no one will complain.
That's because any empty land would probably be entirely unsuitable for a settlement.
Obviously we haven't specified a period in history but this all sounds suspiciously rose-tinted to me.