Not everyone, but you'd have less of a reason to consolidate so many people in a relatively small space.

It's much much cheaper to live outside of major cities. As mentioned elsewhere in this thread, office buildings are a massive source of tax revenue for cities.

This is the entire reason you have this push to bring everyone back.