The trick is that there are two "averages" in play. Let's call them the "attractiveness average" and the "physical average". Your intuition concerns the attractiveness average: you know that there are beautiful people and ugly people, and "average people" must be somewhere in the middle, yes?
But when scientists average faces to create a perceived attractive face, they're averaging together the physical characteristics of each face: distance between the eyes, position of nose and ears on the head, size of mouth, symmetry, etc. The claim is that we have an intuitive, perhaps instinctual, notion of what humans "should" look like and our perception of attractiveness is roughly a measure of conformation to that standard. So an intuitively "average-looking" person is more correctly stated as having a medium amount of deviation from the human mean.