Re:Optimizely Statistics Engine
by dlss on Tuesday January 20, 2015 @11:48AM
(#8917764)
He's saying that as new data arrives, it has to adjust both ways, not only toward the correct answer.
(If that wasn't the case, you could just figure out which was the only direction it would move and then stop collecting data. You've already got your answer)
Re:Optimizely Statistics Engine
by gmu3 on Tuesday January 20, 2015 @12:03PM
(#8917886)
Their graphs don't only adjust up. That one does, but that's because the only really significant downward movement is while their regression is still trailing behind.
Re:Optimizely Statistics Engine
by moultano on Tuesday January 20, 2015 @12:39PM
(#8918133)
It's a statistical significance test not a mean. You expect statistical significance to go up over time. Unless the effect size is zero, it doesn't have an asymptote.
(If that wasn't the case, you could just figure out which was the only direction it would move and then stop collecting data. You've already got your answer)