It's true that a homeowner who pays a higher principal and a lower rate isn't really at a disadvantage if they stay in the home for 30 years (with a 30 year loan). But if rates go up a bit as soon as they need to sell they're going to find far fewer buyers.

And their neighbors will have already found out they had to sell for lower amounts, resulting in some others walking away instead of bringing money to the table at a sale. At thus begins the downward cycle.