But that makes me wonder if the goal should be reframed. Instead of trying to eliminate errors, what if we could change their nature?
The interesting hypothesis to explore, then, is whether a language's grammar can be designed to make an LLM's probabilistic errors fail loudly as obvious syntactic errors, rather than failing silently as subtle, hard-to-spot semantic bugs.
For instance, if a language demands extreme explicitness and has no default behaviors, an LLM's failure to generate the required explicit token becomes a simple compile-time error, not a runtime surprise.
So while we can't "fix" the LLM's core, maybe we can design a grammar that acts as a much safer "harness" for its output.
The problem is that people don't like writing assembler, which is how we got Fortran in the first place.
The fundamental issue, then, is with the human language side of things, not the programming language side. The LLM is useful because it understands regular English, like "What is the difference between 'let' and 'const' in JS?," which is not something that can be expressed in a programming language.
To get the useful feature we want, natural language understanding, we have to accept the unreliable and predictive nature of the entire technique.
If it understands human language enough, it should be able to understand the logic laid out in the documentation mapped to symbols to construct code.
You can also describe your own app's syntax, architecture, function signatures, etc. in markdown files or just in chat and Cursor will write code that conforms to your desired syntax, which definitely doesn't exist in the training set.
Even after that, it will exhibit all the same problems as existing models and other languages. The unreliability of LLMs comes from the way they make predictions, rather than "retrieve" real answers, like a database would. Changing the content and context (your new language) won't change that.