I can see how I could have been in the dark about someone's true feelings if they weren't in a "clique" that I was in. In my exact case though, I felt like the 'clique' wasn't just the people who liked drinking at the bar, because we also had lunches including those non-drinkers, zoom happy hours (2020!), etc. I mean, drinking alcohol itself is optional even at a bar.
You're right that decision-making shouldn't be done by a committee delineated as "just the people who drink together" -- and that could be a real risk if this type of thing isn't being carefully considered.
However, the only guarantee against it than I can think of would be the far worse policy to simply ban all socializing after work, because the same thing could happen if 4 people took up after-work hacky sack and the others weren't interested.
I also totally understand that there's no requirement to drink at a bar; I just don't typically find them to be very fun environments. At least in the ones I've been to post-work hours, they tend to be loud and crowded, and I've always assumed that the only reason people tolerate them as a place to socialize is that drinking makes those aspects less annoying. Imagine if all your coworkers really loved comic books, and after work they always went to hang out in a comic book store, and you had no interest in comic books; you certainly could still go, but without actually being interested in what the venue provides, it's just not going to be particularly fun to hang out there.
The claim that people wouldn't feel excluded and disfavored for not attending but frequent social bonding over drinks because its optional is impossible for you to objectively evaluate. Of course you weren't aware of anyone having anything less than "complete trust" in your boss because of it; why would someone who felt that your boss was favoring those who drank with him tell you? People who perceive they're not part of a special social clique aren't going to go complain to the exact people who are then ones making them feel that way. It's not impossible that you're correct, but it's hard for me to imagine that someone would feel comfortable sharing negative feelings about the boss with you when you seem so confident that no one could possibly have any issues with the arrangement; if I were in that situation, I can't see myself even considering trusting how I really felt about it about it with a coworker who couldn't seem to even conceive of those feelings on their own.