That's unavoidable anywhere population is growing. We don't have a right to buy a house where we grew up. Or maybe those kids can ask their parents to sell their house to them at far below market value. You're asking someone to do that, why not their own parents?

The opposite effect happens in small/shrinking towns. Kids who grow up there and move out for a career can afford to buy a house where they grew up but don't want to. Should these people be forced to move back to their small town so they don't displace the "rightful" residents of the booming city?