Yes. It's a belief based on pseudo-morality, where those who are not privileged are portrayed as being rude uncultured barbarians who, without resorting to extreme violence and oppression, would represent a threat to their pure and pristine existence and social order.
It's the same argument that slave owners used to justify beating slaves to death.
>If people really are trying to kill them, I don't think "panic" is an accurate term and I can't blame them for taking steps to avoid being murdered.
You somehow left out the part where these elites resort to extreme violence and oppression to deprive their neighbors from fulfilling basic needs and instead of helping the community they outright represent a very real and very violent threat to them.
And then, as a feat of cognitive dissonance, these elites cowarding behind their castle walls try to fabricate a moral basis for the violence and oppression they inflict on the very society that permitted their life of privilege.
Looking at say the fall of the USSR it was well connected elites that ended up making vast fortunes.
This is akin to saying "well if some wifes cheat, being a paranoid, jealous husband is perfectly reasonable". It's not that bad things never happen to elites, it's a cognitive disorder because the paranoia is structural, regardless of what actually happens.
It's a disorder because the panic is a function of their social status and their position in society that they know is to a degree unearned (in particular in countries like South Africa or wheverever else the elite is a hereditary class), so in a sense they anticipate the resentment that is rightfully coming their way.
It's like, if you're a habitual liar and thief you also tend to think everyone else is as opportunistic as you are, and sometimes you're right, but that's not why you believe it in the first place.