First, not everything from our body genetic as you say, there is still much to learn there.

But more significantly, intelligence is a very fuzzy concept, much harder to describe and more so define than eye color or height. It is also as much of a cultural concept as things like beauty, charisma, etc... Which is also why it is a more touchy subject than eye color or height: it is much more related to one own ideology.