Combine A/B testing with exponential growth of the user-base and you will only measure newcomers. Medium, advanced, and expert user simply won't matter in your test results. Whether that is what you need depends on the situation.