Unfortunately it would probably not be that great for growth long term and effectively create ghettos away from opportunities and such a concentration would be less able to support services when comprised of the lowest end. Better than homelessness certainly but....
There's no empirical evidence of this happening anywhere else. The European Union and Switzerland are both thriving examples of decentralized governance with totally open borders and free trade, and they don't appear to have such "ghettos". Insofar as the EU has problems, it's that the Member States are starting to get skeptical of its power concentration.
I would even argue that the US today has economic ghettos because it is so centralized — it has a financial center, it has a tech center, it has a couple media centers, and everything outside is barren "flyover country". How many policy Ivy League graduate look to work in Washington DC, as opposed to one of the 50 State capitols?
The only difference is that now they have money to be able to afford a U-Haul and/or plane tickets and/or any other funds necessary to uproot their life to a more economically friendly place.