> selecting the right metrics becomes an art form just as subjective and complicated as designing a good UI in the first place

this is absolutely the truth!

I would say that perhaps what is lacking is a clear vision of the product. Steve jobs didn't use metrics for the initial design of the iphone.

I suspect that as a company grows bigger and bigger, the product loses the initial visionary that created the success in the first place. May be the "founder" (of the product, not necessarily of the actual company) left, or moved on, and what remains is just diluted responsibility between a few different people. Therefore, they resort to metrics, as they have no idea what works, nor have a vision.