why is the LLM-compiler analogy flawed? Is it only because LLM output is non deterministic?

I don't think its just, there's also the fact that if you're working with c or cpp or any systems level language, you typically do know how to read assembly because you've stumbled upon it for some reason, and if you're writing low level programs (which is typically what these languages are used for) you will definitely at some point need to know to read assembly and maybe even write some. But with LLMs the entire field has shifted. You don't need to know anything to write any language and you don't even need to have high level computer science knowledge nowadays to get something that works and the world increasingly just seems to want something that works.

Because you don't have to coax, trick, or guide your compiler into doing the right thing.

Clearly you are not a C++ programmer. :)

Maybe C++ compilers would benefit from asking an LLM to rewrite their messages in a way that makes the point clearer...

But the GP stands.

Compiler: "Here is an exact program. Translate it while preserving its meaning."

LLM code generation: "Here is an intent/specification. Invent code that hopefully satisfies it."

Does the compiler analogy provide value under those terms? I don't think it does. In fact, I think it provides negative value.

We don't need to use tortured analogies to express excitement over these tools.

Besides probabilistic and non reproducible output, programming languages are designed to be unambiguous and explicit, and human language doesn't have that.

For(){} it's normally either undefined or has a specific meaning. "Then iterate and do x" might mean many subtly different things.

Most programmers never deal with a compiler bug in their whole career, and can dismiss the possibility. For LLMs it would be hard to even define what a "compiler bug" would be since there is no specification for English.

Then there's the fact that models generally don't guarantee anything at all. Sonnet can change under your feet.

Models also degrade as the context window gets larger. Compilers handle one line just the same as 20.

I could keep going, there's so many fundamental differences in the process that the analogy only serves to provide a false feeling of security.

I have mentioned it several times lately, but if the analogy was correct, people would be committing prompts and not code. High-level source code gets committed, binaries don't. If prompts were really "just a higher level of abstraction", then there wouldn't be a need for saving the code. Or at least you'd see people publish their prompts and chat history alongside the code.

You can reason with precision about how source code will behave once compiled, and how changes to the code will change the behavior. You can’t reason with precision about AI prompts in that way. This is about more than just determinism, because there are deterministic systems were you still can’t reason usefully about the input-output relationship.