Because the conversion rate was higher from no sign-up to paid. So we were probably selecting more serious trailers, but they were then more likely to pay as well. If it was just the conversion rate from sign-up to paid then you would be right, but we looked at the conversion across the entire funnel.

"More serious trialers" will be more likely to commit by entering an email; people who are not willing to enter an email are obviously not very serious trialers. It's just two different ways of saying the same thing.

No, one implies a causality that the other doesn't.

"More serious trialer" is not a fixed attribute or category that you can select for. A person fits this description based solely on their behavior; they become a more serious trialer by the act of putting in their email address. From that point the causality is the same.

Yes - my point was that the narrative of "why" isn't useful. A/B tests tell you what happened. They do not provide insight into why. Yet everyone has a story they tell to fill in the why, and these stories are harmful to making good decisions.

It's not a causality, it's a pre-filter. So, of the people who passed filter A more are likely to pass filter B.

Today's XKCD was basically about this (in the Alt text)

https://m.xkcd.com/2357/

desktop version: https://xkcd.com/2357/

> We AB tested this