Keep in mind that eugenics, the "old school" version of genetic manipulation, is widely considered a failure, and has a nasty association with certain unsavory leaders that loom large in European/American history (the most notorious one being, um, Hitler). One can say that eugenics has that nasty issue of raw tribalism clouding any potential good judgement. Personally, I would consider the sordid history of eugenics to be a warning sign to everyone in regards to genetic engineering.

The other angle to bring up regarding genetic manipulation of humans, is that genetic manipulation of animals and crops, whether through the old school "selective breeding" or more modern techniques, have not been without unexpected big problems. At the very least, specific health problems have popped up in certain purebreds, and certain bred crops are far more vulnerable to being wiped out by single parasites (eg: our "perfect bananas" have this nasty tendency to keep getting wiped out by the Panama disease). Biodiversity is also the reason that in-breeding of humans is widely looked down upon.

The other ties with the former: it is not impossible for some unsavory leader to take some research, bastardize the science through blinders of tribal instinct, say "Ah! Gene X is the key to IQ!", and do some nasty tribal things "justified" by science. And in the end, get it quite wrong in the end due to lack of biodiversity.