I have tried a similar idea before. I had two experts: one to propose a plan and another to critique it. It worked well, but often required 5 rounds of iteration to get to a great answer. The cost wasn't ideal and GPT4 still performed better on the first go, so I abandoned the idea.

I am curious how this approach compares to one-shot response from a larger model.