yeah I'm over fetching indeed :') A/B testing sucks, they should've let me know it was an experiment, I had high hopes
Yeah, there's definitely room for improvement there, but I feel that it'd only target more advanced users.
I'd like that you could opt-in into being aware of the experiments you are running, and be allowed to opt-out early of experiments when they don't work, or sign-up to run them permanently (as long as they are actually running) if they work great.
Although, letting users know about the experiments might also ruins some experiments as users could start assuming that the experiment really changes things. I imagine you can easily run a dummy experiment and have people talk about how much faster/slower their browser runs with it.
There's a dev running an experiment to whitelist more combinations of the huge space of (Device, CPU, GPU, OS, OSVersion, Browser, BrowserVersion) to use that feature. What would you do after seeing that the metrics show that most of the users on your experiment had a bad experience? In that case you'd want to rollback your experiment right away and figure out what's wrong. You really expected all, if not most of the combinations you selected to actually work well, but it's apparently not the case.