LLMs are autoencoders for sequences. If an LLM can write the code, the entropy value of that code is low. We know that already, most human communication is low entropy, but the LLMs being good at it implies there is a more efficient structure we could be using. All the embeddings are artifacts of structure, but the entire ANN model obfuscates structures it encodes.

Clearly there are better programming languages, closer fit to our actual intents, than the existing ones. The LLM will never show them to us, we need to go make/find them ourselves.