Yea, I think we're in kind of a vicious recursive cycle of imperfect-metric-reinforcement (reward-hacking I suppose, though often implemented in economics as well as code) rather than one of recursive self-improvement in a more holistic sense. Optimization is really good at turning small problems of this nature into big ones more quickly
Relevant: https://www.jaakkoj.com/concepts/doorman-fallacy