I'd be surprised if they actually A/B tested it. Either way, the popup probably did increase the number of email address they got. However, this came at what cost to the bounce rate?