I suspect that A/B testing is doable without collecting user information. Except for the OS, of course. But certainly nothing that's not in server logs.