> "maybe a dashboard isn't the best way to spend your time."

The point of the dashboard is that when you have one, you have actual accessible data from which you can plausibly determine what is the best use of your time.

e.g. If you can see that conversions are already coming in faster than the database is scaling then you know that you need to solve the scaling problem before you worry about A/B testing. Or vice-versa. You don't need to guess (as much).

That's a rather simplistic view and a slippery slope.

Spending time on a pretty dashboard is one of the easiest ways to get drawn into vanity metrics and fapalytics.

Proper monitoring is important, but that usually means completely different things for different departments.

The CEO doesn't need to see database performance charts because he can't make sense of them anyway. Likewise the admin doesn't need to see conversion charts because they are not relevant to his work.

Trying to squeeze unrelated data into a common dashboard often leads to false correlations and entire herds of shaved yaks.

You provided a nice example for a false correlation yourself: The conversion-rate almost never relates to database performance in any meaningful way.

What about a small organization or a one-man e-commerce company where database performance affects conversions, and you're the one-and-only tech-guy?

This was exactly my point.

Small shops have limited tech resources. Database scaling and implementing A/B testing suites can and have competed for those resources.

I think the biggest benefit to the dashboard is (compared to something like A/B testing) it is much less iterative, so with the exception of adding another graph here or there, when it's "done" it's done. If you have the dashboard at all, it continues being useful. I've never heard anyone say, "Finally! We're done with our A/B testing! On to the next to do item..."