News for hackers. Stuff that matters.
Re:A/A testing by alxv on Saturday February 07, 2015 @01:36PM (#9014832)
A/A tests (also known as Null tests) are useful to validate that users are assigned to the control and experiment groups without bias.

Offline resampling methods, like bootstrapping, are better if you're looking to robustly estimate the variance of the experiment.

Re:A/A testing by chucklarge on Saturday February 07, 2015 @01:41PM (#9014863)
Yes, a Null test really should be the first test you run if you're looking to get into A/B testing.

You should also run an ongoing A/A test across your site or app to have confidence your bucketing, data pipeline, stats tests and effect on metrics is working as expected over time.