This is essentially the 'too many of you, just enough of me' argument.

If you want to cap populations on cities, then do so explicitly and not through back-door means. Otherwise you have to let them ebb and flow as people desire.