If that is the case, wouldn't we all just choose Aspen (or La Jolla or Zurich, depending on tastes) ?
It's not obvious to me how recognizing an inherent right to live in any place would be operable ...
We as a society should incentivize people improving their neighborhoods, as right now we're doing the exact opposite in areas with large numbers of renters.
This isn't about outsiders who move to a neighborhood with the goal of having a place to live, this is about investors walking into a neighborhood and viewing it as a financial opportunity.
Landlordism and rent seeking is an inherently immoral thing that we allow to exist in our society, and we need land value taxes alongside reasonable rent controls (like the ones Oregon just passed) to stop these behaviors.
Rent control also has the positive effect that if renters put in effort to improve the economic value of their community and neighborhood, that they wouldn't be displaced in the process. I'd also point out that rent control doesn't always have to happen as a result of low supply, nor does rent control necessarily reduce housing construction should zoning be relaxed in the process (in the era with the highest levels of rent control in Manhattan, was also the era of the highest amount of construction).