I'm talking about running week or month long tests with control and multi test cells containing new functionality, configuration, or code to determine the viability of a single or combination of changes by analyzing statistical output driven by p-value and pre-determined target metrics.

These types of experiments are extremely valuable in uncovering hard-to-find bugs, assuming you have sufficient logging and confidence around your metrics. They let you know a problem exists and roughly where it is in the product. From there you can drill down and investigate your source code until the discrepancy is found.

This makes sense to me. Not the kind of AB testing I had in mind, but fair point. I was thinking more about decision making processes, not operations troubleshooting.

Yes, I know this and I agree. I understand they use the same techniques and terminologies, but this is just not the kinda thing the article is criticizing.