It is very easy to tip an A/B test in the direction you'd like it to go by picking what you measure and handwaving away bad metrics.