Hm… this reminds me of streetlight effect. Not everything that needs improvement/optimization can be measured properly, not everything measurable should be KPI. Tangentially related, I witnessed few times when it caused failure in application of data science to business. The company wasn’t ready, did not have proper data infrastructure, vision or skillset, but still models were built and deployed to no avail.