Something that also tends to be under-appreciated is the constant bias towards successful A/B testing. Successful A/B tests lead to promotions and other positive employee outcomes - negative A/B tests can lead to dismissal in some places. As such people will naturally try to beef up the A/B testing result, or ship a "mixed" result and claim success.