I agree with the second part of your argument, regarding the assertion that LLMs may eventually replace programmers.

However, I don't understand your claim that an LLM acting as a programming assistant "...can’t be reasoned about or meaningfully examined/debugged."

I type something, and Copilot or whatever generates code which I can then examine directly, and choose to accept or reject. That seems much easier to reason about than what's happening inside a compiler, for example.