Not narrowly defined (as in, it is not the government preventing you from expressing a political opinion). But it does have the whiff of a coerced surrendering of one's rights. Suppose you could sign a completely unbreakable "indentured servitude" contract with another person, whereby by a specific payment now, you are required to work for them exclusively for the next 10 years in whichever capacity they so chose (as long as the job is legal), without further pay and without the option to resign (even by returning the money or declaring bankruptcy). The above would probably not be slavery in the legal/historical sense, but I think no country that disallows slavery should allow such contract either.

All contracts have remedies for breach of contract. Note that jail is not a valid remedy. I've signed non-disparagement contracts, where the remedy was I had to pay back the money. Seems quite reasonable to me, and not a violation of my rights.

Such a contract would (almost surely) be barred by the 13th Amendment in the US.

The 13th Amendment language is much broader than the 1st.

"Shall [not] exist" coupled with "Congress shall have the power to enforce this article" is much broader than "Congress shall make no law..."

But traditional free speech rights are founded on the principle that truth is an affirmative defense against (for example) libel or slander cases.

Charging someone with libel or slander is fundamentally different than suing for breach of contract. For example, you don't need an anti-disparagement contract with someone to sue for slander, nor is disparagement necessarily slander.

Now, Congress could pass a law saying the anti-disparagement contracts are unenforceable. But would we really want that? Suppose you hire a nanny to watch your kids, and then she quits and writes a tell-all book about your private, embarrassing personal life? You can bet that nannies for high-profile people have anti-disparagement contracts, and are paid well as a consequence. Which seems quite reasonable to me.

It probably won't happen anyway. But I don't think this is a good argument for these non-disparagement contracts.

You might feel different if you were a celebrity. I know one, and it's sad to see the curtains always drawn on the windows of the house, the paparazzi camped out in front, etc. I could see a nanny being hired and then threatening the celebrity with a tell-all book, or just cashing in on it anyway.

The point is that writing such a book is not always "disparagement".

I don't understand your point. The condition of employment at many places, not just celebrities, is that you don't talk about it outside of work. You get well paid for this discretion. Do you think that is a violation of your rights? I don't, the moment you accept the money.

How about NDAs? Should they be unenforceable?

How about Secret Service agents protecting the President. Should they not have to zip their lips about what they see and hear about what goes on in the White House? Is that a violation of their rights to accept such conditions of employment?

Many disputes are settled with the terms being non-disclosed. Should those be unenforceable, too?

I am not suggesting that the terms not be enforceable, which probably won't happen anyway.