COVID-19 is uniquely terrifying because it is so hard to quarantine. The fatality rate also doesn't factor in the debilitation rate: survivors aren't walking away from this thing unscathed - you have to worry for yourself also. A reasonable amount of panic is healthy, panic is an emotion that we experience for very good evolutionary reasons. Fight or flight often overidentifies threats, but it can underidentify them (e.g. America's official response).

Conspiracy theories aside.