News for hackers. Stuff that matters.
Re:Conservation of Intent: why A/B tests aren’t as effective... by swied on Tuesday July 03, 2018 @05:32PM (#17453765)
A/B testing is a great tool for incrementally improving upon a product design given the correct business strategy. Just don't let the limitations of A/B testing drive how you choose your business strategy. The strategy is ultimately about maximizing total profit, and not necessarily short term profit per user.
Re:Conservation of Intent: why A/B tests aren’t as effective... by citrablue on Tuesday July 03, 2018 @11:10PM (#17455042)
How do you create a discipline around making good decisions for business strategy? I've seen "highest paid person" win, I've seen "Most vocal" win, I've seen "5 versus 1" win.

The testing advocates I've worked with have an attitude of "I don't know what will work - that's why we're testing". I have not seem much of that attitude from most other individuals/teams involved in decision making -- they have preferred to say "this will work", and get angry when I say, "compared to what? how do you know?"

edit - fwiw, I totally agree with understanding the limitations of testing. You have to know what it can be good for. My argument is that it's actually better for long term strategy than most other decision making processes I have experienced, which usually boils down to "gut based".