You make it sound as if it was impossible to have such a marvelous industry without this deep degree of over-optimization. I disagree. The way I see it, with a bit more buffering here and there, it would be mostly the same - perhaps with less bullshit flights that could've been just as well taken by trains (which would be in a better condition, having more business flowing through them), and less middle-men everywhere sucking up the savings from all that optimization.

This is the thing with markets: whatever optimization you make, you won't reap the fruits of it for long. Someone - either competitors or adjacent third parties - will show up to suck out your margin, leaving you mostly as you were, but locking that optimization in as a permanent fixture in the industry. This means that when companies over-optimize, shoot past the optimum "value to profits" ratio and aim for higher margins still, they make the industry permanently (well, until next collapse) worse for everyone.

>The US commercial air transport industry is an absolute marvel. Millions of people moved thousands of miles every single day, mostly at extremely affordable fares. And tons of cargo moved as well

As opposed to...? tons of cargo and millions of people are being moved around everywhere, welcome to the modern world. The US commercial flight industry sucks in the grand scheme of things mostly due to a lack of competition. Flying in the US is expensive compared to Europe/Asia.

I don't claim to know all the price/cost dynamics, but it's mostly the regional low-cost carriers (which people seem to generally hate except for their cheapness) that are cheap in Europe and Asia. Otherwise airfares are pretty similar to the US.