Indeed, contracts can be invalidated if they're abusive. For example, there's a doctrine in contract law called unconscionability. If you buy an airline ticket to Boston, the captain can't say upon landing "Welcome to Boston! You can pay $10,000 to deplane or we'll take you back to where you departed." Or let's say that, when you agree to the terms for a e-commerce site, buried deep in the terms is a clause saying the site gets half the money in your bank account. Sure, both parties might've technically agreed to the contract, but the contract probably isn't enforceable.

*Not a lawyer either, but I've read some introductory contract law stuff. All I know is this: you don't want to rely on the courts to save you, since they generally err on the side of the literal contract, but I don't spend much time worrying about accidentally agreeing to a contract that will sell myself into slavery.