Sure, but I think it is unfortunately that this is the only thing that people talk about.
The reality is that there is a whole bunch of stuff that is really really bad, such as death threats and the like, that will happen whenever there is a hate mob/harassment campaign that was started against someone.
When people try to defend these kinds of online mobs, they tend to focus more on the grey stuff, and they try to ignore the really bad stuff that happens. It is much easier to defend someone just "criticizing" another, instead of the much worse stuff that I've seen happen.
I don't think that someone should be able to distance themselves from the truly horrific stuff that can come out of these kinds of movements. It is much worse than people just be criticize, or whatever. And people need to recognize that those kinds of things happen.
IE, the stuff that happens can often be even worse than the, admittedly still very bad things that you brought up that I agree shouldn't happen either.
And by bringing up the even worse stuff, it gives less room for contrarians to make arguments and claims like "Well, actually, criticism isn't harassment", or "well actually, maybe the person deserved to be the target".
But everyone should condemn violence and harassment from their own ideological cohort.
From the perspective of free speech ideals, "speech" is the expression of an idea; it may be "persuasion" but not "coercion" or "intimidation". Cancellation is both coercive and intimidating by design and by definition, so it falls outside of the boundaries of 'speech'; however, not everything that violates free speech ideals is illegal to the great glee of free speech opponents.