I've had countless arguments with other performers about how rigid one has to be when performing any given classical composition. I'm told over and over again that, for example, Glenn Gould's unorthodox interpretations "aren't musical", but when I press my interlocutors for a justification of that statement, all they can offer is "it's not what the composer intended", which only begs the question. Composers aren't infallible, and they definitely don't always see or hear all the possibilities inherent in their own music.
The whole division-of-labor between composer and performer is, I think, both a strength and a weakness in classical music. It's a strength because it's allowed composers to concentrate on exploring the possibilities of musical ideas and working out something marvelously and intricately crafted in advance. It's a weakness because it encourages this hard-set absolute-textual-fidelity mentality among performers, and discourages them from really leaving their individual mark on a performance (unlike in jazz). The great classical composers are among my heroes, but they weren't gods, and performers shouldn't treat them as such.
Perhaps a violin in an orchestra is supposed to do what they're told, but they are told what to do by the conductor, who himself interprets a composition in the way he wants. And the best conductors are known for their brilliant interpretations, not their fidelity to the composer's intent.
Perhaps classical music has this reputation because there are a lot of really terribly uncreative musicians who would sound awful if given rein to improvise within looser constraints. I have met many musicians who are excellent ordertakers but not what I would call "musical" outside of that.
I think of Conlon Noncarrow, who composed mostly for player pianos. Human beings couldn't play his compositions. He was driven to the player piano by the limitations of human performers.
I don't think that's either architecture or gardening. It's more like community-building.