> You end up hurting both wealthy investors and poor/middle class mortgaged-to-the-hilt homeowners who barely squeezed into the market.

You don't necessarily hurt anyone, you just lower the attractiveness of real estate as an asset class. The yield chasers will move their money into a more attractive asset class, the value investors who buy real estate for price stability or tax reasons would stick around with lower yields.