Maybe you're confusing New Yorkers for everyone else? (In which case, this has everything to do with the NYC culture and nothing to do with academics.) I have met very few industries in which workers will desperately move as far and wide as in academia.
http://www.powells.com/biblio/2-9781400062324-3
She addressed the topic in an essay, "Credentialing vs. Education": http://www.vqronline.org/articles/2004/spring/jacobs-credent...
I don't know about the humanities, but up to about 2009 with a PhD in the sciences from a second-tier institution you could get a tenure-track job at another second-tier university or four-year college. That was the typical career track, because industry recruits only from top institutions at PhD level.
Then the funding crisis struck, and while universities are increasing enrollment (with the students shooting mostly for pre-med at my state school) they are no longer hiring for tenure-track positions, it's now untenured lecturers or part-time adjuncts.
Universities have found out that they can hire at lower salary levels because of plentiful supply, and they do not mind that quality of instruction is suffering. It is suffering because as a lecturer you are moving around every three years looking for a better deal and the first year is spent getting used to the syllabus at the new place, and if you are teaching three sections at three different places to make ends meet as an adjunct you just can't be good.
It's strange that the students don't mind that they are getting less of an education, but they really don't mind.
Unbundling these two things would solve all kinds of problems. That's a pretty big challenge though.
If somehow Harvard could start certifying extra 22 year olds as "just as good as any other Harvard grad" even if they never took a single class there then lots of interesting things might happen.
>> simply getting credentialed (i.e. getting your Ph.D.)
>> does not mean you can count on a tenure track
>> academic faculty position.
This has never been the case. In order to avoid exponential growth, the average professor that trains Ph.D.s can only produce in his/her career about one student who goes on to become a professor that trains Ph.D.s. (ignoring some growth in a particular field). Some schools produce a much larger number of future professors. As a result many smaller schools produce very few.
Listen. At least it's not like many places in Europe (Italy? Germany?) where it's basically an old boys club and someone senior has to literally die before another position is available. At least in north america things are (mostly) still based on merit. You compete for jobs, the cream rises to the top.
Again not to be crass but another pattern I see in modern academia is an unwillingness to move, to take on a job. "I grew up in NYC, my family is here, and goddamnit I want a faculty position here too, not in Ohio or Nebraska or Arizona". I'm sorry but again, wha??
Maybe this sense of entitlement is not adequately dismantled as young graduate students come up through the ranks, that's certainly an issue we can discuss.
Anyway I'm empathetic with the guy in the article, he loves to do X, he is highly educated to do X, and he can't get a top-tier job to do X and so he has to do X-light. Join the club, man. You think everyone who gets an accounting degree expects to work at the top firms? You think everyone who gets a law degree automatically expects to get the top jobs? Of course not. It's competitive. It's a jungle out there.
There's the issue of how long it takes to get credentialed as a PhD, and how it's an extra burden for a 40-something with a young family to not have job security. Sure that's true... but am I missing something? Did someone promise this guy 15 yrs ago when he started on this journey that he would land in a tenure track job? Did someone tell him that it's not competitive? I don't think so.
In fact I'm a bit on the peeved side about this guy, if he does have a young family, what the crap is he doing, why isn't he ensuring a more stable life for his family? For god's sake, get a stable job, do poetry on the side as a hobby like the rest of the world.
Anyway enough ranting, I'm sick of the elite class bellyaching about how they didn't win the lottery even though they bought a ticket.