Well of course they don’t actively worry about your hair, but most certainly it’s one of the factors on whether people find you attractive or not. And that can still have real consequences even if people aren’t “worried about you.”