There are way too countless many stories of people successfully accomplishing tasks by getting an llm to give them a starting point or outline or even a complete solution that only needed a few fixups, hardly any different from debugging your own first draft, to say it doesn't or can't work. That's already sailed.
The fact that it doesn't produce finished correct code, or even appropriate outline for every single case, doesn't seem especially remarkable or damning to me. It is what it is, it's not what it's not.
I find this take surprising. Leaving AI aside entirely, debugging my first draft seems very different than debugging your first draft.
I grant, maybe that is still quite different.
There is something extra to grasp the theory behind someone else's code sometimes.
Because almost every programmer these days has learned by a route that relies on incorporating code on trust. E.g. using someone else's compiler.
> I use code from LLMs on most work days ... not because I trust it, but because I understand it and tested it.
The snag there is that one can rarely verify test coverage is sufficient.
> the same as code from anyone, including from myself six months ago.
But it is not the same. ChatGPT delivers completely untested code. Not even syntax-checked in my case. Would you accept such code from any human?
> Trust doesn't enter into it. Clear and workable solutions do
Plenty of clear and workable solutions are discovered unsafe. I suspect that's much more likely when the code comes from a source such as ChatGPT that delivers code in a completely untested state.