I believe it may seem that "nerds" differ because after all, you're not born with technical knowledge, you get into it by interest and learn it through studying and practicing.
But it is not the technical know-how that nerds get ridiculed for. That's actually about the only part that gets us at least some respect, unless it's downplayed as "just technology".
No, it is the inability to comply to some social norms. For instance, most of us don't care about the latest fashion trends (because we think that fashion is ridiculous). Or, we don't easily pick up on certain subtext signals. It is this non-acceptance that perhaps drives us toward technology in the first place. Because computers don't judge us. Because we understand them, and they understand us.
We are not nerds because we like technology. We like technology because we are nerds.