Turn it around and it's just a matter of not allowing anyone else to infringe on your rights to that extent; to go back to his example maybe you do have the right to sign yourself away to slavery, but nobody has the right to take up that contract.

Everyone uses the "signing yourself into slavery" example, but it's not clear what that means. If you consent to it, then it's not slavery (you can make arguments about "wage slavery," but I think that's a different issue). If you sign a contract to perform labor for some period of time, then change your mind later, the contract likely has some terms for breaking the contract early, and if it doesn't, it's probably not likely to be enforced by any court.

Not being allowed to quit is pretty much the defining aspect of slavery.

A pretty fundamental part of any contract is the cost to a party if they wish to terminate the contract.

That's nothing like slavery then.

Choosing not to exercise a right is completely different from giving up a right.

I can choose to speak while under arrest, but I cannot give up my right to silence. I can choose not to own property, but I cannot give up my right to own it.