They resist the self-promotion to their own peril.
Do you have a portfolio site, sample open source projects, a blog? None of those are golden tickets and can hurt you if done poorly, but they are examples of what I've seen missing when I talk to unemployed (or under-employed) programmers.
Time spent learning how to get a job is probably better spent at this point than learning your craft better. Just short term.
Ageism can lower the odds, but you probably wouldn't be happy in those places anyway.
Another comment mentions the fact that interviewing or marketing yourself may be an issue. It also may be that you're not in the right geographical area, or that you're not interviewing at the right companies, or that your skill set on paper doesn't include the current high-paying buzzwords.
Oddly enough, Android development has been one of the highest demand items for me recently. I do think the full stack development world may be oversaturated, and that 98% of the developers in it are probably too green to do a great job running a project, so I agree with your assessment that most companies have a terrible interview process -- and that they consider programming to be a commodity, where it should be considered some combination of an engineering discipline and a craft that requires a skilled craftsman.
You wouldn't hire an artist whose entire portfolio consisted of collages of magazine photos, and yet companies are hiring programmers who literally can barely get beyond cut-and-paste coding. And you certainly wouldn't hire an unlicensed builder to create a skyscraper -- which is why we have so many companies' backends falling down like a house of cards the moment they get any traction.
Also, what sort of contracts are you signing where you can't even disclose what you did? I've done some consulting work in the past (though admittedly not a lot), and I don't think I've even seen that sort of thing attempted. Obviously there are NDAs on the actual product of the work, but no one's ever attempted to prevent me from claiming I did it.
You work with a constant target on your back. Your bosses have no political capital to defend you in the media, if they aren't marching in lockstep with the demagogues running the budgeting process who are calling your work "government waste", etc. The list goes on.
I'm a 40+ years old life-long programmer with some academic science background, and it's not possible for me to actually get a job. A few years back I thought I'd try to get a full time position, to no avail. The only work that happens for me is freelance work with clients who already know me. It's not an uncomfortable life, but there's also a very distinct ceiling I'm always aware of.
I was instrumental, and often also the de-facto project lead, in launching several multi-million websites and other software, but as a freelancer I'm not really benefitting from these successes. In most cases I can't even legally advertise it. I'm stuck at a medium income level that I feel could dry up very quickly if bad luck strikes.
The job market appears saturated with programmers, and to make matters worse for everyone involved, companies lack the tools and the will to even select applicants properly - I suspect this is partly due to the fact that programming is considered such a low-value skill. At times it feels like it's the corporate equivalent of working as a burger flipper. It's not really considered a dignified position to be in when you're 30. Yet, perversely, the world is built on software.
There is a fundamental misunderstanding - even inside tech companies and SV startups - what the nature of programming-as-a-profession is. In my opinion it should not be some menial work that should only be performed by unskilled youth looking to advance to a "real" job as soon as possible. It's a skilled trade that takes time to develop.