Probably it has to do with socioeconomic classes and level of education. Usually uneducated people or people from rural areas sound, for the lack of a better term, "crude" to big city or urban ears in universally all languages and Spanish is no exception. You have to judge a language or dialect based on how they communicate in official mass communication media like Radio or TV and not some guy you met on the street.