It can be drastically improved in the case of emails by building a Bayesian model tuned to email itself. Most people just blindly apply a testing procedure to their situation, but if you tune the test to your situation you can make do with a LOT fewer samples.
http://www.bayesianwitch.com/blog/2014/bayesian_ab_test.html
It can be drastically improved in the case of emails by building a Bayesian model tuned to email itself. Most people just blindly apply a testing procedure to their situation, but if you tune the test to your situation you can make do with a LOT fewer samples.