News for hackers. Stuff that matters.
Re:Microsoft attributes $80-$100 million dollars in revenue ... by unblogworthy on Thursday March 18, 2010 @12:33PM (#1201722)
It think they'll need to do some A/B/A testing to be sure.

Switch back to the original shade only for those that experienced the new shade. Maybe they'll see another multi-million dollar increase? That way you could further prove that it's indeed the color that made the difference and not simply the fact that the page was changed at all.

Added:

The Hawthorne effect is a form of reactivity whereby subjects improve an aspect of their behavior being experimentally measured simply in response to the fact that they are being studied, not in response to any particular experimental manipulation.

http://en.wikipedia.org/wiki/Hawthorne_effect

Re:Microsoft attributes $80-$100 million dollars in revenue ... by bemmu on Thursday March 18, 2010 @02:49PM (#1202131)
Personally I like to A/B test with new users. But I guess on an open website like Bing (instead of one with logins) you won't be able to really know if someone has visited it before.