Yep. I've seen "rigorous" A/B testing regimes set up based on a p<0.05 requirement that then went straight into "we'll keep running this until we reach significance" and "this one's clearly trending towards significance, we should just implement it now" nonsense.