>If I invest 20 years of my income in a property that someone has stolen, that is a significant loss to me in human terms as well as financially.

You've also paid for your primary residence. But on top of that, you need it to live in. So it's a worse situation, generally speaking.