I think a lot of it is style more than the specific clothes. If you're wearing a tie that you didn't tie well, and doesn't match your outfit, that's going to look bad on you literally and figuratively. A lot of people "overdress" but still don't dress well. I'm not expecting you to show up in a suit, but if you decide on your own to wear a suit to the interview, it better be a clean suit you look good in.