I think this is related to my fundamental problem with A/B testing: when people A/B test individual variables they'll take a greedy optimization that gets stuck in local minima. The other problem is that you only understand the change in metrics you measure and things like "satisfaction" are often lost.