The descriptions I've read about a/b testing make it sound a lot like null hypothesis significance testing, which has a well earned reputation in the social sciences. And you have to buy the things that you're testing, either from a vendor, or by employing people who assemble the stuff.

Advertising signals are normally too weak to be detected by standard A/B tests. Additionally, most traditional advertisers do not run A/B tests. Their agencies especially do not run such tests. Source: https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=lewi...