More realistically in the current USA, you know you are extremely privileged when you're more afraid of being labeled as a Nazi than actually being harmed by one.

Or, you have an unusually strong reason to fear being labelled a Nazi, like you are a vocal racist who attends public demonstrations carrying or dressed in Nazi accoutrements or attire.

Or, you just have irrational relative levels of fear between those two outcomes.