Most programmers aren't. And it's often a tug-of-war between designers and programmers to gain respect for their roles: have a designer always come later in the process and you'll get more of a limited traditional visual designer; have one always come earlier in the process and hand down visual specs like they're stone tablets, now you've got old school art directors from the Mad Men era and programmers are treated like code monkeys (still mostly the case at ad agencies today, actually); have one on your team that can design upfront while doing usability testing to validate features, all on a tight schedule, now you're talking about today's interaction designers, but they don't come cheap and most startups have a hard time landing one.

Moral of the story: Always have a design up front. I didn't have a design so the structure of my code was incorrect for the "visual" the other person had in mind, so it broke. If that visual is solid, especially when working with/for another person, then fewer mistakes will be made down the road.