Not only symmetry, but also the proportions. People are evolutionary wired to use symmetry and proportions of faces and bodies to estimate health and survival/reproduction "value". This is not the only factor, of course, but a huge one.

I remember in biology we learned that symmetry is an indicator of good genetics, so it makes sense that we are biased to symmetry since higher fitness/survivability.