News for hackers. Stuff that matters.
Re:Microsoft attributes $80-$100 million dollars in revenue ... by nandemo on Friday March 19, 2010 @09:16AM (#1204044)
Why would they have to change it for an entire year? Maybe it's enough to test it for 1 day, or 10 minutes for all I know. If it improved clicking rate and revenue by 9%(+-1%) and after 1 year they've got $1 billion of revenue, then you could say that 8~10% of that billion likely came from Magic Blue.
Re:Microsoft attributes $80-$100 million dollars in revenue ... by mtarnovan on Friday March 19, 2010 @02:47PM (#1204918)
You want to sample for 10 minutes or 1 day and extrapolate for 1 year ? The results would be meaningless.

Even if you sample 1 continuous month you could have badly flawed results (for example you might have sampled only early adopters, which might behave differently; and that's just one example of the many, many factors that can make signal indistinguishable from noise). Now I'm sure that the guys working on Bing A/B testing know what they are doing, but I just can't believe a change in link color would bring a "$80-$100 million dollars a year increase" (from a total of ~2000 million $/year revenue which I extrapolate from this http://www.microsoft.com/msft/earnings/fy10/earn_rel_q2_10.m... but which includes all revenue of the Microsoft Online Service Business - I don't know how much of that is generated by Bing)