To reply to your middle paragraph, you'd probably want a longitudinal cohort study rather than an A/B test. A well-designed one can provide pretty conclusive indications in situations where A/B testing (or, testing with a double blind control as the epi community might call it) isn't possible.