We should have two distinct, independent LLMs: one generates the code, the other one the tests.
perfect use case for GANs (generative adversarial network, consisting of (at least) a generator and a discriminator / judge) isn't it? (iiuc)
Do you also hire two different kinds of programmers - one that never wrote a test in their life, and is not allowed to write anything other than production code, and second that never ever wrote anything other than tests, and is only ever allowed to write tests?
It makes no sense to have "two distinct, independent LLMs" - two general-purpose tools - to do what is the same task. It might make sense to have two separate prompts.
I may be biased, I am working with a codebase written in copilot and I have seen tests that check if dictionary literals have the value that was entered in them, or that the functions with a certain type indeed return objects of that type.