Re your edit: Yeah based on all the replies so far I think I must've given the wrong impression in my original post. Too much emotion, makes it hard to interpret reality.
I think this maybe gets to the core of the problem, that it's not really to do with the social setting per se, but my attachment with it, and the expectations that go along with it.
Just like here on HN, I easily get carried away with idealizing the place and then become overbearing, too eager to impress. Same online or offline. Except maybe offline, what I've realized, is that people easily get overwhelmed by my eagerness, so they kinda just step back from it all, not wanting to get involved. Whereas online, I can't really overwhelm anyone cuz it's all thrown into the air, and anyone who cares will respond, and those who don't just move on. That's my theory anyway.
But in general, I think the context around us really does matter a lot, like you mentioned. Not just the types of people we meet, but the expectations we/I set up with it.
Another thought that I'm starting to explore, is that perhaps I'm the one that isn't smart enough or well-read enough to carry a conversation with anyone about anything. Whereas online/HN I can pick the topics I know about and go contribute there.
Lots to think about. Me + social life = confusion to the max. And I've been thinking about it and trying things out a lot, and still can't figure it out.
That being said, it's possible to punish yourself way too much for what you did or didn't say, dissecting every conversation afterwards. I was a very shy kid. I had a lot of thoughts, but I didn't speak up. I would describe myself as someone on the spectrum who figured out how to communicate using a different methodology than most people; it didn't come naturally. When suddenly around 15 or 16 I began to get confidence, it loosened up my tongue, maybe way too much. At certain points, especially in my 20s, I remember thinking that I was going too far (trying to impress, or to make a point) and sounding like an idiot. But I was extremely hard on myself about this, like I think you are being. I would have a great conversation with someone and then wake up the next day feeling suicidal because I was sure they thought I was stupid or annoying. Even when this bore no relationship to reality. Even with people who liked me and wanted to hang out.
One thing that helped me was a year I spent in therapy - specifically, existential therapy. It helped me see that I'm just what I am, we're all just what we are. No one's perfect, and no one spends anywhere near as much time analyzing my social faux pas as I do. That at most, people might think I'm eccentric or a curiosity. And being so hard on myself took away the pleasure of relating to other people, put too much focus on me, and made me too defensive. Why was I always worried about being judged - why did I assume that everyone else was judging me and yet I always accepted them for who they were? So much comes out of your own sense of self-doubt. It colors your interactions in ways you hardly realize, and it can lead to a dark place when you're trapped in your own head that way.
Firstly, don't be embarrassed. It's good that you want to engage and be stimulated. But by the same token, don't expect too much in terms of personal love & understanding here. I've been a coder for almost 30 years and I still don't understand half of the things people post here, and it's tough to find people of similar minds.
What I think you're looking for is being in a social setting that stimulates you. And it sucks to say that it's so damn hard now to do that. When I was a kid in the 90s, I spent all night in art/coffeehouses talking with people from cops to drug dealers, writers and artists to Costco employees. That culture is gone. It's been replaced by a cheap knockoff version online, but you never see or know the people, so you don't really get the same experience of closing out a place at 4am with friends you know you'll see again tomorrow or the next night.
My GF grew up in Texas and thinks of herself as someone who escaped being a "normie". Her whole family is super religious. She has "normie" friends who plan an annual trip to Disney World, stuff like that. She gets most of her intellectual stimulation from learning how to code and talking to me and a few of her not-normal friends, weirdos like me who she picked up over time. And what I do alone and we do together is, we just plant ourselves outside a bar and start chatting away. Or reading a book. You'd be amazed at the people I meet. Now... I live in Portland, OR. When we go out to bars near her place in the deep countryside, the chance of a really interesting conversation gets a lot slimmer, and we spend a lot more time talking about cars and kids than like, philosophy or international relations (or antifascism) the way we can in Portland. So one big thing I've learned is that where you are really matters. The right city or the right group of friends/neighbors can help you blossom. The wrong place can just keep you down, make you hide away or give up trying. No matter how friendly you are. I'm also very friendly, and easily embarrassed when I'm too friendly. I'm not white - or not perceived as white, anyway, so it's hard to tell sometimes what people are thinking. You know genuine people when you see 'em.
Don't fetishize HN. It's a high level dork conversation, that's all. Honestly, it's dangerous to think too much of it, because a lot of people here aren't worth the salt. Wherever you live, there are definitely cool dorks nearby, you just need to meet them.
I think if you sit outside Starbucks or a bar with a book and eavesdrop on other people's conversations and just - at an opportune time if you hear something interesting, don't be afraid to say "hey, I heard what you were talking about, mind if I ask something?" Well. This takes practice but it's how I met most of the best people in my life. Not online.
[edit] Just reading your reply to someone else that you work in a university bookstore makes me realize you probably come into contact with more interesting people than I do every day. I have literally no one in my real life who has a clue what my job involves, except for one or two other people who take laptops to the local bars who trade war stories with me once in awhile. The reason I'm on HN is literally because as a freelancer it's the only place I get to talk shop. Everyone needs to talk shop sometimes.