Somewhere in there I think is ego gone awry; why write for what you think audiences want to see? That's such a cynical corruption of art.
I suppose it's fear of your movie gross failing to exceed your production budget, and you retire as a legendary director anyways. Ah, that doesn't seem too bad. Well worth the risk. But, no.
I'll start: Spielberg.
I would argue with this, though: "Avatar and Titanic presumably aspired to be excellent films as well."
No, I don't think they did. I think by now Cameron had internalized the screenwriter's gospel of character arcs and what audiences (presumably) want to see, and regurgitated it. "Excellent" to him meant "what people will pay for."