You've just made an unsupported leap of logic. The question is who is silencing who and in what context. Parents may discipline their children. Church staff may eject anyone they like. Soliciting sex is a crime. None of this means a mob can rightfully silence unpopular, legal speech. And certainly not in the name of freedom.