This is a contextual multi-armed bandit. You pass in visitor attributes like location, referrer (did they come from google, a specific ad or directly), are they on mobile or desktop, what operating system, what time of day, day of week, etc, to automatically show the variation most likely to convert each unique user. You can implement this with an ML model, e.g. a gradient boosted decision tree, that is continuously trained on the incoming data. You can set a goal metric to optimize for, like revenue. Like any multi-armed bandit, you'd have an explore / exploit ratio so you can adjust based on new data, e.g. new visitor segments or visitors start preferring different variations. So you don't even need to get rid of variations as they're shown less if they perform worse but if they start performing better, they're shown more again. This is a bit like how ad platforms work. Instead of A/B testing what ad works best, they use ML to match the best ad, out of millions of possible options, to each unique user. This is the optimal approach to landing page optimization. We're building this as a platform and have had great results.

What's interesting about the platform we're building is you don't have to "remove the associated code" to remove variations, as all the variations are in the platform itself. E.g. let's say you're testing a headline. Rather than set up variations of the headline in your code, you make the whole headline configurable in the platform, a bit like how a CMS does. Once that's done, you can set up the headline variations in the platform for the ML to pick from, then add / remove them without touching the code.