In fact, my feeling is that you are asking the wrong question. I mean that what matters is not the quantity of agents. Because one agent is sometimes too random (or has too much freedom), whereas on the other side, having plenty of agents is too strict. The rule is to allow each agent to have freedom inside, and to be strict on the frontiers.

You can imagine that you have a real team of people (not AI). They are all very smart but very junior, with no experience in the problem and context that you know well... Provide them enough freedom to exploit their intelligence, but provide the right directives, scope, and rules to do the minimum of the job.

For example, define clear boundaries for your agents: specify what they receive as input, the expected outcome, and what is strictly forbidden to do.

Hope it helps