My untested hypothesis is that your parents and peers expect you to act a certain way (based on your appearance) and many people accept these subtle nudges without resistance. So, it basically becomes a self-fulfilling prophecy.