One key reason for all this pain is very simple - too much competition. If we had large institutions like the old bell labs (being funded the same way), that would create demand for fresh PhDs and reduce some of the stress.
Academia is also considered by almost everyone I know to be a one way gate. If you get out, you can't come back in (unless you have achieved massive success and popularity on the outside). This is a sad state of affairs.
All this said, I think there are some very good results of getting a PhD. My social circle is full of ridiculously smart people. I think I'm trained to be unsatisfied until I better understand things, and I have more critical thinking today than when I started grad school.
It is about the teaching euphemism: students are rather used as cheap labor for exploitation to be disposed of, when "too old" (read expensive: family etc.). I guess there is also a legal reason why trainees and interns are on the rise as opposed to students and initial probation periods.
The other thing we fail at as academics is helping trainees understand and take advantage of non-academic tracks after grad school, without attaching a stigma of failure to them. There are many, many ways to succeed, have a rewarding and exciting career, and contribute to society as a non-academic with a graduate degree... not to mention the higher salaries! Too many academics treat their trainees as failures if they even initiate this conversation. It needs to stop.
Imagine if students entering med school, who wanted to be pediatric neurosurgeons, for example (sub-sub-speciality requiring many post-med school years of study) found out at the end of all their training, at age 35 or 40, that oops, perhaps only 25% of them will ever get the job they want?