BUT, when you're the size of Google, or Facebook, or Microsoft, Things look a bit different.
Most people are far more conservative than they realise, but also, perversely, adapt quite quickly to change if they experience it day-to-day. This means that if an interface has enough inertia, then a change that performs badly in user testing, can actually quite quickly be accepted by people. (PROVIDED it's actually better).
In the first example, every change to the Facebook UI is met by a barrage of criticism and hatred, but they seem to actually make good decisions, so the changes get absorbed by users, and things continue.
Having said that, Microsoft struggle, because they often make changes that are fundamentally bad decisions, and have to back-down. For example, the Office ribbon took a while to be accepted, but is /generally/ liked nowadays by people, because it was functionally a good change (yes, some people will disagree, but the majority wins on this front). Taking broad user testing results on the ribbon would probably have stopped microsoft from progressing with it. On the other hand, the 'Metro'/Tile UI in windows 8 was an actual usability disaster, and has forced them to revert to a more common pattern until someone can design new metaphors that actually improve usability in windows.
Finding test subjects that aren't used to, and comfortable with, the google font/design styles, to give objective feedback on a change, is really hard.
[ I didn't mention first time round, but I /really/ don't like the new logo, that lowercase g makes me cringe inside every time I see it. ]
The evolutionary, data-driven approach will never yield a Nest thermostat, iPhone, Helvetica, or flag of New Mexico. We have no idea how to even begin to write a fitness function for good design.
Formalizing that process and using externally-collected data is just de-black-boxing something that happens in a black-box form in the professional's mind.
You point out the need to pick the right measurement, and I agree, but it could be as simple as "What's the effect on conversion?" It has to be something! You can't just say "We're re-designing because I'm a HMI genius."
Until then, the best known way to design something nice is to hire a talented designer and trust their instincts.
...for now, given today's technology, but I wouldn't bet on it long term. History is full of examples of things we once thought only humans could do.
But once many candidate designs are available to choose from, what better a way is there to unambiguously decide on the best besides testing and measuring?
But at that point, why not just cut out the middleman, forget the tests, and let the artists be artists?
See also: Footnote 3 of http://www.paulgraham.com/colleges.html#f3n
The trick is: there is no global "better" — only 'better', given a specific group of users.
Which users though? Running a million 40 year olds from Iowa through your website flow will generate a widely different result from running a million 13 year olds in Atlanta through the same process.
Google is still stuck in the "one design for everybody in the world" mindset, so they have to be appealing to everybody, which naturally makes them bland.
success metrics you're using
Success metrics! The paradox of these hyper-optimization methods is they largely apply in hyper-trivial situations. Once you are big enough, you just tell people how things will be. They don't get to have a say anymore.
And you can't rely on past data to create new trends because that data doesn't even exist.
Data said that children's books don't sell. But then Harry Potter happened. Data also said that people loved phones with keyboards. Then the iPhone happened.