Run fewer, better A/B tests

Re:Run fewer, better A/B tests by eximius on Saturday June 26, 2021 @12:14PM (#27643121)

I'm going to stick with multiarm bandit testing.

[ Parent | on HN ]

Re:Run fewer, better A/B tests by nxpnsv on Saturday June 26, 2021 @01:13PM (#27643717)

This is a much better approach...

[ Parent | on HN ]

Re:Run fewer, better A/B tests by gingerlime on Saturday June 26, 2021 @04:22PM (#27645488)

What tools/frameworks are you using for running and analysing results?

[ Parent | on HN ]

Re:Run fewer, better A/B tests by eximius on Saturday June 26, 2021 @09:19PM (#27647637)

At my previous job when it was relevant, I wrote something in house.

A higher order component would pick which variant at runtime (cause we had problems with SSR, or that would be more appropriate). Cached the picks in cookies.

In house charting and probability calculations to determine what P(X>Y) was for each experiment pair. Then we'd just manually prune them occasionally (since the bad ones weren't being displayed, timeliness didn't much matter). Periodically re-introduce old variants by hand if we thought it was worth it.

[ Parent | on HN ]