But why does your right to say your opinion out loud trump someone else's right not to feel threatened or harassed?

That's what you're basically saying with "no one should feel threatened and if someone does then that is their line".