Re:A/B testing gets misused to juice metrics in the short term
by josefx on Tuesday July 12, 2022 @10:47AM
(#32070796)
But does AB testing provide the only or even best metric for that? It probably is the cheapest way requiring the least engagement with the lab mice.
[ Parent
| on HN ]
Re:A/B testing gets misused to juice metrics in the short term
by treis on Tuesday July 12, 2022 @06:04PM
(#32076276)
AB testing is how you isolate a change and measure the impact. It's the only real way to be able to associate cause and effect. Best you can do otherwise is measure something over time while making changes. You can try and correlate changes with outcomes but it's hard to be sure the change is what drove the outcome.
Re:A/B testing gets misused to juice metrics in the short term
by oba2311 on Wednesday July 13, 2022 @07:10AM
(#32081098)
That sounds pretty accurate. Anyone who's gone through some econometric classes would know that split testing is (almost) the only way to get to (close to) causality - all the rest is black magic called correlation which can also lead you to the conclusion that Nicolas Cage's movies lead to people drowning:
https://www.thenationalnews.com/uae/nicolas-cage-movies-link....
https://www.wnycstudios.org/podcasts/otm/articles/spurious-c...