I'm not psychologically trained, so take this with a grain of salt, but I've met exactly one person that I think was a narcissist. He was a jerk, but mostly because he was utterly baffled by any concept that didn't involve him in the center. It was an attribute completely separate from his confidence. I feel like it is really hard to explain if you haven't met someone like that - I was routinely gobsmacked by his reactions - every human interaction I'd had to date had not prepared me for it.
Meanwhile, I have tons of anxiety and often cover it up with false confidence, and our reactions to similar events were dramatically different.
The part that made me realized I was dealing with mentally diseased people is that they truly believe the crap they throw at you. Their lies become an undeniable truth as soon as they spell it out loud.