Let's pretend your mum was opening a diner, and she wants to sell the best fries in town. She has a good recipe, but knows with a little tweaking, it could be great.

She has a few regulars who come for lunch each day, so decides to see what they think of little changes she makes.

She starts by adjusting the amount of salt, a little more for some tables, a little less for others. She notices sometimes tables order more refills of their bowl of fries, and even stay longer (buying drinks, etc...).

Rather than making bigger changes and directly asking customers their preference ("do you prefer our new super-fries or our classic-fries?"), she's decided to focus on more subtle (often not very noticeable) changes, and most importantly, watched the reactions or changes in behavior to influence any longer term changes.

(imagine everyone she asked said they like some new seasoning, but now only order a single, small serving. maybe the original is actually preferred?)

I feel like making a video from this response. This is such a good analogy.

The Coke/Pepsi challenge. What matters is not which one is Coke or Pepsi, but which one consumers preferred.

We are going to divide up the people who come to our site. Half of them will see one version of the site and half will see another version of the site.

Behind the scenes, we are collecting a lot of site metrics. After the test we will compare the metrics and see which version of the site scored higher.