The fact that they sometimes can't means you can't have someone who is not a software engineer. Otherwise there is no way to verify whether the LLM wrote correct code or not.

Exactly. Expertise is frequently required to steer, correct, rework, know what is good and what isn't. This happens 500 times a day on both the micro level and the architecture level.

I feel like this depends on the kind of project/work to some degree. Lots of software engineering time is spent on very simple websites, apps and other projects that LLMs have been well trained on. I suspect if you want a static site with a contact form or a CMS theme/plugin, you could probably get a usable result by having some non-technical person plug some prompts into an AI.

They wouldn't be able to verify if the code is correct, but based on my testing, if the work is simple enough, it probably wouldn't matter much.

There are a lot of companies whose modus operandi is basically "build the same basic site 300 times, except with a different colour scheme and logo".

Yeah configuring whitelabel products for customer’s brand should be right in LLM:s ballpark.

Unless they can produce the right thing 100% of the time, they require an observer with enough technical merit and experience to validate

Well we would think that wouldn’t we but businesses can be run by people for whose business ”works mostly” is good enough.