So, I think a key challenge is that modern programming languages do 2 things simultaneously.
1. They provide direct instruction to computers.
2. They document precise human intent.
Even if the need for part 1 goes away, part 2 will always be with us. I would expect that if natural language programming becomes a thing, that a dialect forms - akin to legalese - that embodies best practices for precisely documenting human intent.
Sort of like how the SQL syntax seems designed for business admin folks to use, but us programmers ended up using it.
So it seems that a declarative programming language would be a suitable target for LLM to generate.
Maybe someday there would be a popular, shared IR across models, like SQL does to relational database.
So a bit like `create-react-app` or `dotnet new` or such, but now powered by AI?*
This has existed since at least 1992 (Visual C++) and probably before that (Borland Pascal?).
Not sure if you played with chatGPT, but you can ALREADY ask it to do this and it will give you something that works 50% of the time.