That's just your own private imaginary wall that you built up in your mind. Nobody else had a hand in putting it there.

Just tear it down and resist the urge to be always available. It gets much easier over time, I promise.