Re:Architecture update: leaner, meaner and faster A/B tests
by paraschopra on Monday April 05, 2010 @12:47PM
(#1242303)
Yes, you are probably true. But I have seen users permanently implementing a variation on the slightest hint of improvement, so it is better to take a strict stance because they are going to ignore it anyway.
Ben has IIRC previously written, and I agree, that interaction effects are typically minimal in practice unless you have catastrophic implementation choices. Since you're taking care of the implementation and it isn't catastrophic, you can safely ignore them. Don't advise your users against using your software in a manner likely to increase the value they get out of it. Interaction effects are worth an asterix in the documentation, but not much more.