I agree that it ought to be, but to a lot of people, those rights end when they involve behavior outside unspoken social norms. The concept of rights and freedoms has always brushed up against that limit. There's been constant change in what those norms are, but changing them has never been easy.