> I also think we need a high level formal specification language to tell agents what we expect them to do.

Let’s make that specification Turing complete while at it.

Jokes aside, IMO it will be a good natural progression. Specify the problem statement in LLM specification, generate the code in Go/Rust whatever is the language of your choice and review the generated code to make sure it adheres to the architecture/design principles that you have set.

It absolutely should be Turing complete. I want to formally specify some constraints/invariants that any generated code has to meet, like very high level test cases.

It doesn't have to be a new language. I'm sure some existing language can be used to create a DSL that serves this purpose.

It can obviously never be complete. Some parts of the spec will always have to be natural language if we want to make the best use of LLMs.

Maybe we can have Large Logic Models instead, and they could have formalized keywords with rigid meanings? Like IF, WHILE and FOREACH maybe. Or even ASYNC if you want to be modern about it.

I do think that AI models should get better at logic. But if code generators are supposed to be tools, we have to tell them what to do. I'm not sure what combination of languages is best for that purpose.

It would be so much easier if we could precisely specify what we wanted, without all the double meanings, slang and general ambiguity that comes from using a natural language.

If only there was an entire class of well-studied languages which don't have any such ambiguity. They'd be perfect for programming LLMs! We could call them "programming languages" perhaps.

But what we want is a lot of ambiguity on the implementation side and some targeted ambiguity on the specification side where appropriate.