Hand the average person a highly specialized work on mathematics, biology, or computer science and they'll quickly explain to you that they lack the prerequisite training or reading to understand it. Hand them a philosophical tract or a novel of literary merit and they seem more likely to either attempt to struggle through it, ignoring things they don't 'get' to come up with a impoverished sense of the work, or they'll claim the language is too (and unnecessarily) foreboding and dense, but not necessarily the subject matter--there's no sense that the difficulty lies in lack of training, but rather in something inherit in the work (conceived as a flaw or virtue). The onus is on the work to be comprehensible, rather than on the reader to comprehend--yet the obverse relation is oft upheld for texts in the sciences (in the rooms of average opinion. Specialists, of course, feel quite differently).
There seems to be less general acknowledgement of the fact that dedication to the tradition and assimilation of historical concepts is just as requisite in the humanities as it is in the sciences.
It's odd how certain subjects are victim to the full fury of uninformed opinion while others are immune. Ask anyone of his politics and he'll (usually) soon explain his view and why its correct, all the while finding the fact that he hasn't studied any political history, political science, political economy, or political philosophy totally irrelevant. Ask the same fellow his thoughts on a mathematical conjecture and he'll quickly scramble his way out of the conversation. It probably comes back to the mild religiosity our culture maintains about the sciences--its seems a profane thing for mere laymen to opine on the word from on high. Meanwhile the things of men are open to all cant and blasphemy.
The sad thing about this framework, is there's no obvious place for stuff that's not only abstract, but perhaps orthogonal to the economic sphere. So nobody gets taught philosophy. I think this lets everybody down, because with stuff you learn at school, you generally either learn it's hard, or you learn some things about it. With philosophy, you learn neither. So you think it's easy, and you know nothing about it.
I guess there's also some compounding effects from the prestige that gets attached to various subjects - I think philosophy is a weird case though, since the better an academic or scientist is, the more likely they are to take philosophy seriously.
The whole thing used to really bug me - but then, when people take philosophy really seriously (which seems to me, the history of christianity - a lot of people getting beaten to death over kind of dry philosophical trivia), it generally leads to worse philosophy. So everybody making the subject a target of shame and ridicule, making it out to be a ridiculous, worthless and pretentious occupation - all this actually is to the benefit of good thinking, in the end, since you won't get anybody philosophizing with ulterior motives.