Oh, there are certainly exceptions to this, Gary Bernhardt (whom I highly respect for giving excellent talks) wears exactly the attire I'm describing.

But we all have implicit biases, and it wouldn't be my gut feeling to trust a person who does not present as 'one of my group' like this.. even though I'm aware of the bias.