I'm not advocating design by committee. I'm advocating justification with data. The designers should propose candidate designs. But the decision about which one to go with (or whether to redesign at all) should be supported by whether going with it furthers the business's goals.

You point out the need to pick the right measurement, and I agree, but it could be as simple as "What's the effect on conversion?" It has to be something! You can't just say "We're re-designing because I'm a HMI genius."