I tend to have no problem saying no to things that I think are bullshit, I have no problem skipping out on extra "team building" efforts or declining to attend the happy hour. I let my work speak for itself and I figure companies keep me around for that, and coworkers know they can come to me with any issues they might be having.
But I have never purposefully taken a coworker out to lunch or asked about their life outside of work (except for when we're on a call before anyone else and they start the conversation), and yet I've had just as much success as people who I've watched do that.
That's not to say that I don't get to know my coworkers I just let it happen organically as we're building things and solving problems together rather than going out of my way to make it happen.
I take the sycophant model of being easy to work with, but I also make sure I don't learn anything outside my silo so that I don't get a lot of work and can interview prep.