It is likely that they view the speakers as advocates of ideas that limit the individual freedoms of others. Making it about free speech is a bit of a misdirection. When that speech furthers a system they disagree with, they are likely to make a stand against that speech.