If you weren't abusive or neglectful, they'll probably be fine. Let's say you were a little bit distant and distracted and they end up a little bit 'avoidant'. That means they're not going to rely on other people for emotional support when they're feeling down. That doesn't mean they're going to be emotionally crippled for the rest of their life.