I think it comes from liking people a priori, but having a habituated expectation that the people we know will take from us (our happiness, our time, etc.)--so that one can't feel safe to be oneself with those one knows. IMO this expectation typically comes from childhood and can best be mitigated via mindfulness.