The right wing has always had its fringe nutjobs and even terrorists; these seem (or seemed, until recently) to be trending down in severity and frequency, and everyone across the spectrum has typically condemned right-wing political violence. We're pretty good at marginalizing right-wing groups and mitigating their violence to the point where the odds of being impacted by right-wing violence were super low.

That feels less scary to me than watching hundreds of people descend on my neighborhood or a nearby neighborhood and set it alight or otherwise destroy it followed by the media and cultural elites (never mind colleagues, bosses, etc) tacitly approving of the violence (if not outright celebrating it--memorably a former ESPN reporter tweeted "Burn it all down!" and then when the violence approached his own neighborhood, "Get these animals TF out of my neighborhood"). This isn't to invalidate anyone's fear of right-wing violence; we all have different experiences and perspectives; I'm only sharing mine.

Note that we don't have to choose between left and right wing violence. We can and should condemn all kinds of political violence--indeed, it seems like every spike in the popularity of right-wing identity politics lags some much larger spike in left-wing identity politics, and every spike in right wing violence follows some spike in left wing violence and so on (e.g., you don't have proud boys without antifa). If I'm right, then it's kind of good news, because it means we can curb right wing violence and extremism simply by holding the left to the same standard.