Yes but that changes the real difference between conversion rates of A and B. Even though trend won't get affected but actual numbers will. Not that serious but given that this problem is easily avoidable one may want to not announce the test.