To that end, there is no "right to be unmolested", and I would underline that particularly for--given the context of this thread--conceived-of rights that effectively serve to provide legal supremacy over other people. (Of which we have many.) "Done nothing wrong" can mean very different things to very different people, and while I am not in favor of ex post facto convictions or anything of the like, I am in favor of amending the rules to the game to be more equitable and, I hold, thus better for the aggregate of humanity--and enforcing them to that end.
So, no. The general will sets the rules of the road, and dissenters can comply (and seek to change that general will if they so choose; part of the societal privilege to try to change that general will is the societal responsibility to obey it when it breaks against you), accept the consequences of civil disobedience (the abrogation of the aforementioned responsibility in hopes of demonstrating injustice to encourage a change in that general will), or leave (fold your hand). As it happens, there are cases where I'd do the second and a couple where I'd do the third.
Can I choose not to leave, if I respect other people's natural rights?
My issue about leaving is that if I have done nothing wrong, I shouldn't have to flee from others who wish to violate my rights. Instead, they should respect my right to be unmolested.
Interested in your thoughts, if you wish to expand on this topic.