Whether or not they are free to punch you has no basis in whether or not your speech is protected in that instance. And sometimes they are free to punch you.

If you say you're going to kill me (not protected speech), and a reasonable person would believe you, I can hit you in self defense (affirmative defense for criminal battery).

> And sometimes they are free to punch you.

Not if your speech is protected, that's the point. And that's also what cancel culture, doxxing etc is about. They're not going after people announcing plans to kill the President or shoot up a school.

Free speech without the government enforcing the physical integrity of the speaker is meaningless.

> Not if your speech is protected, that's the point.

Actually, the point is that your speech is protected from the government (and with certain specific exceptions). Your speech is specifically, emphatically not protected from non-government actors.

But that's not related to free speech as much as it is to assault being illegal in general.