The same way that we generally speaking don't respect land rights of native people who were displaced by settlers hundreds of years ago, but we do respect your land rights, if I were to displace you, by 'settling' into your house.

There's a gradient between the two, and the line between them tends to be drawn rather arbitrarily - but it does exist.