I think the EU is a good example of how the border as 'a thing that prevents people from moving about' has been eliminated (via the Schengen Agreement) while it still maintains them when it comes to nation states, voting and so on. I think that's exactly the right way to do it - I think people should be able to move about, but it's also important to let people vote on a small enough scale that it is meaningful, and within a small enough group to share a culture and world-view.