I get your point, but don't you think it is somewhat complicated?
When you are the main dev in such a project, would you be willing to give this huge responsibility to some guy who might loose interest after a few weeks? What then?
Also, I have seen people doing graphical sketches of possible UI's and people think nice - but it is a whole different thing to actually implement this (especially with so little money involved), when the to do list involves 10 million other things.
Also, usually the problem is the designers are not coders and quite often the code base is not designed in a way, where you can just swap out the UI for a new one.
But yes, I'd love to see more collaboration of designers with a vision and coders.
My point is that people don't understand the necessity for a "vision guy" to direct UX in any program of appreciable size. And it's not like these kinds of people don't exist, or that they'd be unwilling to help. People just don't seem to even WANT them taking on these tasks on their projects.
And that's a real shame, because many of these projects are technical wonders that are borderline unusable for want of a loving touch.