News for hackers. Stuff that matters.
Re:Interpreting A/B test results: false positives and statis... by bertil on Saturday October 30, 2021 @04:58AM (#29047203)
It depends on the area of Netflix, but for most recommendation that can fit on a list, Netflix seems to use interleaved testing:

* they create two recommendation lists based on distinct principles or parameters;

* they pick a first (with an A/B test);

* mix those with ‘schoolyard rules’: each side picks the next candidate that wasn’t picked already, creating a new, alternating list.

* They show that to the user and track whether you picked an even or odd item on that recommendation, indicating whether A or B is preferable.

So most like, you are seeing A and B.

There are no comments on this comment yet.