Some pretty easy rules of thumb, assuming you have a decent grasp on your economics. Look at it as a "low hanging fruit" optimization problem -- do you put resources into running a test (+ opportunity cost for lost sales), or into something else?

Suppose you have 10k monthly sessions with a 0.5% conversion rate (50 conversions). How many more customers would you need in order to prioritize running a test? If 55 conversions in a given month means you crush important KPIs, then that's probably worth testing -- you just need a 10% lift.*

Also keep in mind that running A/B tests (1 control, 1 treatment) is suboptimal. That tests, "does this beat what I have now?" The more important question is "what is my best option?".

OTOH, if other things like messaging and product are stable, you can test a smaller traffic site by leaving it running longer.

My rough estimate is 100 conversion events in the time the test runs. So if I have 100 conversion events in 1 month, it may make sense to run a 2-3 option + 1 control test for 1 month.

(You can also test much larger things than buttons. For startups, I like to suggest trying out positioning or value statements and seeing how visitors respond!)

* however, it'll take a long time for you to reach statistical confidence for a 10% lift in rate, with only 50 conversion events across all tests.