The rock is mine because my family bought land a few hundred years ago, and the previous owner thought the rock was ugly and let it for us.

> If the rock means little to you, why should you have it rather than the cult that actually values it?

The rock means a lot to me, because it reminds me of my ancestors. I value it, but I just don't have a billion dollars to attach to it as its value.

> What if the cult wants to preserve the rock but you want to destroy it even though that will cause great pain to those who care about it?

I dunno. I think it is my rock and I can do with it what I want. But I don't want to destroy it, I just want to keep it where it has been for the past 200 years, near my family home's fireplace.

> Why should you have permanent ownership of a rock just because your parents had it?

Because that's how ownership works. If anyone can come take anything from you at any point, then I don't think you can ever be considered to own something.

Maybe you can say you don't have to fully abolish property. But it remains especially hard to predict what second order behaviors any particular implementation would bring about, and what steady state would result from them.

Per your example, if the expected ruling of society would be that gp doesn't get to keep the rock because they don't "care" about it... well, maybe people in their position would suddenly find themselves caring about the rock very greatly. Not even cynically -- if the choice presented to you is "value or no value", the limbic system suddenly becomes very persuasive.