Re:A/A testing
by chucklarge on Saturday February 07, 2015 @01:41PM
(#9014863)
Yes, a Null test really should be the first test you run if you're looking to get into A/B testing.
You should also run an ongoing A/A test across your site or app to have confidence your bucketing, data pipeline, stats tests and effect on metrics is working as expected over time.
Offline resampling methods, like bootstrapping, are better if you're looking to robustly estimate the variance of the experiment.