It's definitely a big ballpark, but I think Dunbar's Number is a good place to start. If you have managers spending 10% or less of their lives on management, I don't think the organization will be very healthy. Management should be a high commitment, high compensation role.

It's also definitely a upper limit rather than lower limit. Big bureaucracies with many layers of management and small teams can work well, but no one can really individually manage 1000 subordinates.