I just can't agree that's fundamentally true, unless you have in-home monitoring set up. Swizzle that to "You should be able to leave your house for two weeks without a water main break flooding it" or "You should be able to leave your house for two weeks without a gas leak blowing it up."
Real estate ownership implies obligation. It's one of the reasons so many people choose to just skip out on that entirely and rent their residence.
> Frankly, if you tolerate this kind of behavior against rich people, it is eventually going to effect you as well
Frankly, if I tolerate leaving people unhoused long enough and in large enough numbers that it effects me, it should effect me. Those are probably fellow citizens and fellow neighbors, and if not, they're still fellow human beings. Adequate housing is a human right as per the Universal Declaration of Human Rights.
Down that train of thought is "taxation is theft," and it's not a good place to be.
(Cards on the table, I've come from the other side of the story: absentee landlord with undeveloped property sitting next to a relative's home, and the neverending fight to keep the dead trees on that property from crushing said home in a storm. The process to get a property that is owned by someone five states over who is holding it to be sold for 2x the market value and hasn't seen the property in fifteen years declared a nuisance property is atrociously long.
Owning property implies an obligation to the place where it is owned. The law should have more teeth to enforce those obligations.)
But I see that as no excuse to take my property away if I'm doing everything right. And the squatters can't tell who is absent and who is just on vacation. We have to deal with this problem seriously.
I do live in a neighborhood with a large unhoused neighbor population, who are well aware of squatter rights and looking for opportunities, so its not just a "well-heeled" problem for us. Frankly, if you tolerate this kind of behavior against rich people, it is eventually going to effect you as well (they just get bolder and bolder).