Indeed. When I was advocating for A/B testing at the last company I worked for, I tried hard to teach people that every test should have a sane hypothesis behind it based on some kind of usability theory. Not just “let’s try changing the header font color to red and see what happens!”