Walmart has a lot of good basic clothes but a lot of people -- even people who aren't particularly rich -- just wouldn't dream of buying clothes there because it has blue-collar associations. This is really just one element of the rich tapestry of signals a person gives you as to their social class (clothes and language are probably the most distinctive and immediately noticeable signifiers).