I think most LLM codegen successes is due to their translation abilities, which is what transformers were designed to do in the first place. Software developers usually solve problems in human language (or maybe a sketch) with general “white collar reasoning abilities” that most of us honed in college, regardless of our major. The translation to Python or whatever is often quite routine. A human developer’s software-specific problem-solving skills are needed for questions involving state, unfamiliar algorithms, “simple” quantitative reasoning, newer programming languages, etc... all of which LLM codegen is pretty bad at.