There's a sweet spot I think. Wearing a suit? Probably a business guy, not to be trusted. But raggedy T-Shirt? Ah, that's just unprofessional. The ideal male fashion icon: Mr. Rogers, classy but not trying too hard, everybody trusts a nice sweater with a collared shirt under.