Another boss I had any a different job told me he fired someone for not partying with everyone else. If you went to these parties and didn't drink, the boss would harass you all night.
I agree with you that many people are just self conscious, but it's not exclusively that way.
The tl;dr is that people who are insecure will attempt to rationalize their feelings, and this frequently includes feelings of judgement and persecution from others where none exists.
Unless being drunk is your idea of fun (in which case it seems doubtful you'd be a non-drinker), I cannot understand how drinking liquid A at a party is any more boring than liquid B.
In fact a graph of my enjoyment as a non-drinker (in my experience) would probably be inverse to a graph of the enjoyment of the people getting progressively more shit-faced.
Is this seriously a thing in the US? Feeling like you have to drink if you're surrounded by people drinking? I keep hearing stories like that. Nobody is going to judge you for not consuming alcohol - if anyone does, they're certainly not people I'd want to hang out with regardless of whether drinks are involved.
It doesn't hurt not to give a fuck, once in a while.