But if I don't consent to the "social contract" called private property rights, then your exercise of the rights that you've claimed that I don't recognize seems equally coercive.

I happen to believe (in agreement with you, I think) that private property rights look more useful (in a benefit-to-humanity way) than the rights that most people who use the term "social contract" think of, but I don't believe there's a clear philosophical difference between the two.