Re:Sixpack: Language-Agnostic A/B Testing
by sutterbomb on Wednesday August 28, 2013 @02:01PM
(#6291579)
+1 on the unreported control. It's a common use case to start an A/B test at a small test percentage, then ramp up over time. Without an unreported control, you can't do that and have valid samples for each group.
(E.g. your sample mix will be invalid if you start at 10 test / 90 control then ramp up to 30/70. But if you start 10 test / 10 control / 80 unreported, then ramp to 30/30/40, your samples will be valid.)
(E.g. your sample mix will be invalid if you start at 10 test / 90 control then ramp up to 30/70. But if you start 10 test / 10 control / 80 unreported, then ramp to 30/30/40, your samples will be valid.)