Using your example of "obviously good", lets say something seems "obviously good", so you launch it, and after a week, you see that it converts worse or better. That's an A/B test, where A is before the change and B is what is after. You don't need software for that, you don't need to do on-the-fly A/B testing, you can just launch.

Given a site has existed for a while, it has a pretty known A for most things you'd want to test. Doing A/B tests on small elements seems may indeed not be worth the extraneous costs of time etc.