If you're doing A/B testing where you continuously watch the results, and conclude your test as soon as you see the numbers significantly deviate, what you're doing is more-or-less equivalent to what's happening in the comic, only it's the number of jelly beans that you vary instead of their color. Not exactly the same, since in the numbers case your draws from the random variable aren't independent, but still.
For starters, consider this xkcd comic: https://xkcd.com/882/
If you're doing A/B testing where you continuously watch the results, and conclude your test as soon as you see the numbers significantly deviate, what you're doing is more-or-less equivalent to what's happening in the comic, only it's the number of jelly beans that you vary instead of their color. Not exactly the same, since in the numbers case your draws from the random variable aren't independent, but still.