"People with Down's syndrome are not less intelligent than average" is the sort of thing that you can only say in an ultra-progressive environment. It would not be taken seriously by most people. Under any reasonable definition of intelligence, those with Down's syndrome have less of it on average.
Your observation about multiple intelligences isn't novel or profound. The interesting thing about intelligence is that different kinds of intelligence are positively correlated (SAT math scores correlate with SAT verbal scores. Musical ability is correlated with mathematical ability. Etc. etc.)
At this point you've just recited the standard college freshman talking points, which tells me that this isn't a subject that you're curious about. https://en.wikipedia.org/wiki/G_factor_(psychometrics)
I don't even think a statement like "It seems clear to me that environment dictates more than heritability of genetics" is intelligible. The idea of dividing it into X% genetics versus (100 - X)% environment betrays an incredibly naive understanding of the question.
It's a bit more than that. If you look at all the different measure of intelligence that people have come up with like vocabulary, finding patterns in shapes, SAT scores, etc, and do a factor analysis there seems to be more or less a single factor behind the correlation. Some of these, like vocabulary, have an additional large effect from training. Some, like vocabulary and SAT verbal scores, have a strong correlation independent of the the rest of these varied scores. But in general there seems to be one factor, the hypothetical g, that explains why everything is correlated together. And it seems to explain part of other test results too that don't seem to be what we would normally consider intelligence, like the classic reflex test that measures how long it takes you to press a button after a light turns on.
IQ tests try to measure this 'g' by using a number of different subtests chosen to not respond strongly to practice. But these, like all tests, have a certain amount of noise and shouldn't be taken as absolute truth.
In regards to arguing that genetic defects cause lower intelligence, you have to define intelligence. The rate at which certain, particular categories of knowledge are learned? The speed at which knowledge is utilized? Are autistic savants the most intelligent people to have ever lived? People with down syndrome seem adept at reading and responding to body language, in my experience. Precise empathy is certainly a form of intelligence. I think our intelligences are diffuse. In the abscence of how biological data relates to these, we are just pattern matching.
After that, you have to ascertain what is even a factor in those intelligences: the total pool of genes we can draw from, the ways the genes express themselves on or off, or the delicate mosh-pit of biochemistry weaving our personality together. There are too many variables to control for, variables that aren't understood. It seems clear to me that environment dictates more than heritability of genetics. How much more is up for debate, but I'm not confident genes will be adequately contextualized in my lifetime.