There are a few CEOs that seem to truly make a difference, and maybe those would be worth the money, but for the most part companies are grossly overpaying for people who are mediocre to terrible at the job and will deploy the golden parachute in a couple of years to go fail at a different company.
I think that there are a lot of people that are cut out to be CEO, but most of them are already working as CEO - of a small business, or household, or one-man consultancy. There are very few people that are cut out to be CEO of the particular big corporation that is hiring for CEO. If you take the Ribbonfarm essay at face value, the quality that qualifies a person to be CEO is that their natural personality is in total alignment with the direction that the organization needs to go, and so they can merely be themselves, loudly and brashly, and they will attract followers that take the organization in that direction. There are just a handful of people, oftentimes one or none, whose natural personality is perfectly aligned with where the board wants to take the company. When you find that one, you should pay top dollar to get and retain them, because you may not find another.
Child of old CEO, internal promotion, or executive have wildly different applicant pools but you can find great or terrible CEO’s chosen based on each.
I’ve worked with several CEOs and countless senior executives, both at very large companies and small, who had varying degrees of competency. I don’t think this is generally true. Being successful at that job entails a specialized skill set that is rarely evident in middle management. What you are “managing” at the most senior levels of large organizations is very different than as a middle manager, it becomes an entirely different kind of role. The vast majority of people aren’t even able to competently run a small company or startup, never mind a large enterprise.
I think very few people are really cut out for that role. It is a case where demand exceeds supply, which drives up prices. There are many mediocre people in those roles because supply is so scarce.