The thing that frustrates me is that if you speak out against safetyism, some people will ignore what you're saying and just declare you a safety threat.

It'd be nice if we made more decisions using cost/benefit analysis instead of emotive fearmongering.

I actually think 2001 is a good talking point, because by now there is widespread agreement that it represents a case of fearmongering getting out of control -- with the benefit of hindsight, it's clear that people who e.g. spoke out against TSA excesses were not, in fact, aiding and abetting terrorists. But it seems like analogous arguments "aiding and abetting" arguments are currently being made in a few other domains.

Agreed. Cowardice and recklessness are both evils, and it’s possible reject both.