A lot of math detail, but a naive reader reading it will still make the mistake outlined in http://www.evanmiller.org/how-not-to-run-an-ab-test.html. And a less naive reader will still be left with no guidance on how to make real-world decisions if you weren't so lucky as to get a strong statistical result.

Here is a much, much simpler strategy that I can suggest for business users.

1. Figure out the longest that you can afford to run your test for.

2. Figure out the number of conversions N you expect in that time.

3. Start running your test, randomly splitting visitors into A and B.

4. Stop the test as soon as one version has sqrt(N) more conversions than the other. Else wait until you get to N conversions between them, and go with what is ahead.

Here are some comments on the procedure.

Stopping before N/2 total conversions is roughly the same certainty as stopping at 95% confidence. Stopping after that is an admission that you are crossing your fingers and going with an educated guess, and it is not feasible to collect enough data to get a better answer.

If one version has a conversion rate which is (1 + 2/sqrt(n)) better than the other, you pretty reliably choose right. The flip side of this is that you'll make a lot of mistakes if you get much below that threshold. If those potential errors are too big, then A/B testing is not going to work well for you because you don't have enough data for statistics.