Re:The art of programming and why I won't use LLM
by shafyy on Sunday August 25, 2024 @05:19PM
(#41351529)
Compilers are deterministic
Re:The art of programming and why I won't use LLM
by codebje on Sunday August 25, 2024 @06:09PM
(#41351923)
This is in general only true for either trivial toy compilers or ones which have gone to lengths to have reproducible builds. GCC for instance uses a randomised branch prediction model in some circumstances.
Re:The art of programming and why I won't use LLM
by shafyy on Tuesday August 27, 2024 @05:09PM
(#41373072)
Ok, but my understanding is that that they are mostly deterministic. And that there are initiatives like Reproducible Builds (https://reproducible-builds.org) that try to move even more in that direction.
Re:The art of programming and why I won't use LLM
by Wowfunhappy on Tuesday August 27, 2024 @08:41PM
(#41374885)
But what does "mostly" mean? You can compile the same code twice and literally get two different binaries. The bits don't match.
Sure, those collections of bits tend to do exactly the same thing when executed, but that's is in some sense a subjective evaluation.
---
Szundi said in a sibling comment that I was "completely [missing] the point on purpose" by bringing up compiler determinism. I think that's fair, but it's also why I opened my post by saying "I agree [with the parent], but I want to try to define the language better." Most compilers in use today are literally not deterministic, but they are deterministic in a different sense, which is useful as a comparison point to LLMs. Well, which sense? What is the fundamental quality that makes a compiler more predictable?
I'd like to try to find the correct words, because I don't think we have them yet.
Re:The art of programming and why I won't use LLM
by shafyy on Wednesday September 04, 2024 @01:12PM
(#41448132)
I'm not an compiler expert, not by far. But my understanding is that if you compile the same code on the same machine for the same target, you'll get the same bits. Only minor things like timestamps that are sometimes introduced might differ. In this sense, maybe they are not deterministic. But I think it's fair to classify them as "determinstic" compared to LLMs.
It's not that LLMs aren't deterministic, because neither are many compilers.
It's also not that LLMs produce incorrect output, because compilers do that to, sometimes.
But when a compiler produces the wrong output, it's because either (1) there's a logic error in my code, or (2) there's a logic error in the compiler†, and I can drill down and figure out what's going on (or enlist someone to help me) to fix the problem.
Let's say I tell an LLM to write a algorithm, and it produces broken code. Why didn't my prompt work? How do I fix it? Can anyone ever actually know? And what did I learn from the experience?
---
† Or I guess there could be a hardware bug. Whatever. I'm going to blame the compiler because it needs to produce bytes that work on my silicon regardless of whether the silicon makes sense.