When you include the full spectrum of LGBTQI you probably have to cut that down to no more than 95%, possibly less.
When you say "for the vast majority" you presume they operate in isolation, but they don't. Most people will end up knowing at least one person that doesn't conform to the traditional gender patterns, which means it's a problem everyone faces.
So that's the explanation why most people don't "get" or even think about the distinction, but they know about gays and lesbians.
Treating sex and gender separately makes it seem like they just accidentally coincide in case of 99.x% of the people.
One needs a very special kind of schooling and/or social circle to even imagine there's a difference. It seems to be very limited to (some) Americans.
Some people suffer from phantom limb syndrome. The pain is real, but the limb is not. We've not yet reached a point where the politically correct thing to do is to pretend otherwise.
There would only be PC pressure to pretend a trans man has a penis if "having a penis" were a vital component of being a man, which it is not. In any case, the circumstances under which one finds themselves having to confirm the presence/character of another persons genitals usually either fall outside of or supersede the jurisdiction of "political correctness".
Also, most languages have no such distinction. For example in my native language Hungarian we either use the English word "gender" or say (the equivalent of) "social sex" and "biological sex".
It's a niche thing. I understand that some people feel sensitive about sex/gender or the differences between transsexual/transgender/transvestite/queer/crossdresser/drag-queen etc. But for the vast majority of the people this just isn't a topic of interest.