News for hackers. Stuff that matters.
Re:Most AB-tests will fail by RyanZAG on Wednesday June 19, 2013 @12:02PM (#5906312)
Couple points there:

#1 The article is very vague - the Obama team may have actually made a very common mistake and stopped the tests early when they hit a significant level (they were in a big rush at the time, they had days to get everything working, not months). We don't know anything about samples sizes, etc.

#2 The main detail they talk about changing was changing from a movie to a still picture. This is a lot different than changing a button's color - many people will see a movie and close the page if they are in a quiet environment such as work or a library.

So it could very well contradict the A/B testing article by Wired and still be correct - or it could be wrong. Contradiction itself is not a red flag here in something as new as A/B web testing.

There are no comments on this comment yet.