Because location is not fungible. Desirable places to live stay desirable many times because high income people compete with each other to live there, hence when incomes go up, so do the prices they’re willing to pay. The good schools with all the children of educated parents are there, as are the facilities, stores, restaurants, airports, mass transit, weather, parks, etc. The supply of places with a combination of all of those things is very limited compared to demand.

Places that don’t have many high income people competing don’t see this kind of appreciation.