There are lots of stupid musicians, just not any I want to hear. But be careful comparing writing to performing. Yes, you can be a mindless automaton and still sound pretty good. But composers are generally intellectual, and in my experience the set of good composers who are not blindingly intelligent is very small.
So there is this idea of different arts being more or less accessible to the general population.