Why impossible? It's just a mindset. People do switch careers at >40 years, it's far from impossible.
The social pressure can be overwhelming, unless you're not super-super strong, mentally. We cannot assume the average person to be so strong.
Why is the social pressure overwhelming? Socially sometimes it seems that it's better to crawl into a cave and never be seen in your poverty, than be seen as a 40-45+ year old doing janitorial work (or other kinds of jobs most people find degrading, despite the fact that to your face they will deny it), by your friends and family. Also a lot of people can't accept the downturn in their lives. That's how you get alcoholism, depression, drug abuse. It's much harder to wake up and say: I've failed all these years, I need to start over.
If we go by anecdotes, I knew only two persons who switched careers at mid life successfully: one was a woman and the other a man and both had a spouse who brought decent money to the household and was supportive.
Now that I'm 40 and poor again (lost my job due to cutbacks), it is impossible for me to recover. Getting new jobs are much harder and people are pretty discriminative towards my age. Nowadays, even employers choose younger people for part-time jobs, since I'm over the hill agewise and overqualified.