Re:Twitter 2.0: Our continued commitment to the public conve...
by solarkraft on Thursday December 01, 2022 @05:51AM
(#33814678)
Think of the AI optimizing for a stated goal while ignoring implied constraints (e.g. eradicating humanity to stop wars).
This can be the case for A/B testing. Sure, you can increase ad clicks by 30% ... if you trick the user into clicking it through a carefully timed layout jump.
I think GP's argumentation may go in this direction. I'd probably not say A/B testing is the problem itself, it is a tool after all, but I could imagine it's sometimes not used very well.
Another point: Spotify's core flow changes so much (feels like almost daily) that I've lost all confidence in using it.
Re:Twitter 2.0: Our continued commitment to the public conve...
by P5fRxh5kUvp2th on Thursday December 01, 2022 @10:41AM
(#33817726)
To make it easier for everyone to understand the other posters point...
a double blind test can help you determine the most effective way to cause pain to a monkey, but it will never answer the question of whether you should be doing so.
In my experience, they mostly just catch bugs. Stuff like "hmm, our much better looking signup flow underperforms... oh, the form is broken on Safari." That kind of thing.