While it is true that many decisions people make aren't rational, and it may even be true that most decisions most people make aren't entirely rational, the claim that the whole software market, which is under huge selective pressure, manages to make irrationally wrong decisions in a consistently biased way is quite extraordinary and highly unlikely. What is more likely is that the decisions are largely rational, just don't correspond to your preferences. It's like VHS vs. Betamax story. Fans of the latter thought that the preference for the former was irrational because of the inferior picture quality, but VHS was superior in other respects - recording time - that mattered more to more people.
I was programming military applications in Ada in the nineties (also not memory-safe, BTW) and I can tell you we had very good reasons to switch to C++ at the time, even from a software correctness perspective (I'm not saying C++ still retains those particular advantages today).
If you think so many people who compete with each other make a decision you think is obviously irrational, it's likely that you're missing some information.
Cyber Security itself is an example of this. It may seem rational to want guarantees of security for the entire supply chain. But that simply isn't possible in reality.
A professional effort is the judicious application of resources to the highest priorities. That includes care in design and testing. Applications built with C and C++ are running everywhere around the world, every minute of every day.
0. https://stackoverflow.com/questions/28426191/how-to-specify-...
1. https://hackage.haskell.org/package/range-0.3.0.2/docs/Data-...
Ada does not have curly braces.
Stephen Bourne, author of the Bourne shell, used macros to make C memory unsafe language look like Algo 68 memory safe language.
The root of the problem is measurement. Speed is one of the few dimensions of software that is trivially quantifiable, so it becomes the yardstick for everything. This is textbook McNamara Fallacy[1]: what is easy to measure becomes what is measured, and what is not easily measured is erased from the calculus. See developer velocity, cognitive overhead, maintainability, and joy. It's the same fallacy that McNamara made in Vietnam and Rumsfield made in the War on Terror so at least they're in good company.
This singular focus distorts decisions around language choice, especially among the inexperienced, who haven't yet learned to recognize trade-offs or to value the intangibles software process. Like you said, humans are irrational, but this is one particularly spectacular dimension of that irrationality.
I find it hard to reconcile this with the actual observed trend of all software getting slower and more memory intensive over time
Application performance is a very important factor. To ignore it is foolish.