I guess we are currently in the special situation, that we as human programmers can understand the output of a coding LLM. That's because programming languages are designed to be human readable. And we had an incentive to learn those languages.

I imagine that machine learning powered coding will evolve to an even blacker box, than it is today: It will transform requirements to CPU instructions (or GPU instructions, netlists, ...). Why bother to follow those indirections, that are just convenience layers for those weak carbon units (urgh)?

Simultaneously, automation will likely lead to fewer skilled programmers in the future, because there will be fewer incentives to become one.

Together those effects could lead to a situation where we are condemned to just watch.