Stock price is most certainly not the only dimension by which to measure CEO performance. In fact relying solely on that metric is exactly what fosters our current culture of absurd CEO compensation packages.

Both "Operational Impact" and "Leadership effectiveness" are equally important considerations[1].

Quoting from the source:

"Operational impact. Operational impact refers to the CEO’s influence on the company’s effectiveness in operational areas, such as customer satisfaction, new product introduction, or productivity enhancement, and how well the firm implements its strategy. Operational impact measures often give a better indication of a company’s underlying potential to create value because they are directly related to the immediate stock price, which is subject to market-wide volatility. While still subject to external and internal forces outside of the CEO’s immediate control, this type of performance is more closely related to the CEO’s actions.

Leadership effectiveness. Leadership effectiveness addresses how well the CEO carries out his or her responsibilities, both in terms of executing specific role responsibilities—identifying a successor, meeting with key customers and investors, developing a long-term strategy—and the quality of those actions—communicating with external stakeholders, energizing the organization, and gaining the confidence of investors.Rivero and Nadler (2003)."

Source:

[1] https://saylordotorg.github.io/text_corporate-governance/s10...