> Having to test every assertation sounds like a not particularly useful application, and the more variables there are the more it seems to be about throwing completely random things at the wall and hoping it works

That should be fully automated.

Instead of anchoring on "how do I test what ChatGPT gives me?", think "Pretend I'm Ansys Inc.*, how would I build a platform that combines an LLM to figure out what to make in the first place, with all our existing suite of simulation systems, to design a product that actually meets those requirements?"

* Real company which does real sim software