You can support free speech in the abstract, end up defending certain people whose views you hate, and still come up with a good amount of respect. The ACLU is able to maintain respect despite helping some pretty terrible people in court.

There's just a cohort of people who claim to be in favor of free speech, but also use it as a defense to associate themselves with people they really don't need to. Even the worst people in the world need _a_ lawyer, your local fascist doesn't need a booster on Twitter. There's a spectrum and subjectivity here of course.

"Free speech" has turned into a fun little bad faith thing to throw into arguments where it (for most people) doesn't belong. And even for fellow travelers, these people arguing in bad faith tend to throw in some other stupid garbage into their arguments! So it gets a bad rap, as an indicator that an argument is about to get stupid.