That's one of the reasons that COL factors, by themselves, don't mean an awful lot. If I were to move into a city it's clearly not even meaningful to talk about the price of the equivalent property to my exurban house on a fair bit of land. The relevant comparison is to what I would/could realistically and at least somewhat contentedly live in.

As I said elsewhere you need to make that after-tax housing cost comparison. Aside from a few other potential factors like private schools, most of the other purely cost deltas are going to be relatively modest.