I think asking someone to live in a home they own 6 months a year in the same city as their property(ies) satisfies 'fairness'. It's important to allow people to live their lives, but over speculation hurts many people to a much larger extent.