"but most people will benefit (every single landlord, lots of businesses that will sell stuff, and all the people that would move and get a job there)"

That's far, far from "most people". Landlords, for instance, would benefit, due to being able to charge higher rents. But everyone else in that town that rents would not benefit. And I dare say there are more renters than there are landlords.