So if I'm your fiance and you suddenly find out that, say, I am a member of the KKK, you can't decide not to marry me (that would after all be a consequence beyond simply disagreeing)?

Obviously that would be absurd. As an individual I'm entirely within my rights to respond to the views of another beyond simply disagreeing and debating with them.

It seems entirely reasonable to me that I would be able to choose not to spend time with someone with views I found abhorrent.

So how about choosing to work (or not work) on a voluntary project being run by someone whose views I disagreed with?