But if the LLM can use decompilers in the process? It already uses many tools (grep, python, sed, etc...); so why not a decompiler and then take it from there?

I've seen this approach already in practice: LLM+decompiler give better code than the original code that the binary was compiled from.

I'd imagine it's just a lot less token efficient than having it write the code in a high level language.

I think you're right, but rather surprisingly I have some experimental data on this question that is inconclusive at best. My hobby project is symbolasm, which is intended to close exactly the gaps you describe: by naming variables and introducing types, it re-adds much of the context that assembly language erases. I ran a quick'n'dirty experiment to see if it would make the LLM more efficient at writing code than raw assembler.

The task was rewriting an MP3 decoder from Rust into ARM assembly language, Cortex-M33 flavor specifically, optimizing for both speed and code size. The output is approx 3k instructions. The tl;dr is that Opus 5.5 nailed the task, whether writing assembly or symbolasm. It took approximately 2x tokens for the latter, which I think is largely explained by the fact that it was dealing with a novel language nowhere represented in its training set.

The quality of the result was comparable; the raw asm was slightly faster and slightly less dense, which I consider insignificant because I think you can get the same effect just by tuning the prompt. In particular, if I were re-running the experiment, I might set a strict code size budget and then ask it to optimize for speed within that budget, allowing comparison on a single scalar.

The experiment was compelling in another way: the code was 3x denser and 1.5x faster than -O3 output from Rust. So for certain applications, going to assembler will be worth it, especially as the cost of generating the code goes to zero.

[1]: https://github.com/raphlinus/symbolasm

And there is the following critical advantage: you decouple your code from any compilers. Some compilers are sane of course, and their devs are doing an amazing job (look at cproc/qbe/etc), unfortunately the mainstream ones are c++ abominations (gcc and clang/llvm) which we should depart from.

If all this AI thingy is true, the decompiling and recompiling directly from the machine code is a sort of proof that what's the most important is the data model. Additionnaly, when I get into some software which I did not write, I must focus on this data model in order to understand it, and guess what, the direct memory one from 'human' assembly/C is usually much better than those giga-convoluted object oriented abstracted ones (I think the worst one was llvm, usually everything c++ is).

Some are generating "human" assembly code for their everyday software (I am following some of those projects I found here, on HN). Again, if all that is true, the killer feature: you could generate "human" assembly for several ISAs fitting one data model, for instance the IP UNlocked RISC-V (vectorized or not)!