That is true for existing, middle class and below housing for sure. Whenever interest rates go up, we see the exact effect you are describing.

it's not the whole story however. In order to increase Supply, it has to be profitable to build new homes. If building new homes is not profitable, none will be built, and the price of existing homes will be forced upward until it matches the cost of new construction.

So I agree with you that access to debt is a very important factor, but I disagree that it is more important than demand when you look at the whole picture.