Re:Will LLMs Eclipse the Classic Code Diff Algorithms?
by baristaGeek on Thursday November 09, 2023 @04:06PM
(#38211309)
In a world where you can pass a whole git repo as part of the context window of a model, wee see potential not only to reduce hallucination but on diffing based on a company's code style
Re:Will LLMs Eclipse the Classic Code Diff Algorithms?
by DarkPlayer on Thursday November 09, 2023 @03:45PM
(#38211001)
I think LLMs will have a hard time replacing traditional diff algorithms. You want your diff to be reliable and predictable. It shouldn't omit relevant changes. This is not really a strength of LLMs.
I think a system that parses the code and uses verifiable rules to hide invariant changes would have a much easier time being adopted. I may be biased since I work on SemanticDiff (think difftastic but for VS Code and GitHub) and have implemented such rules myself. There have been several times where I've thought about implementing a new rule, only to find out that some obscure edge case would not be handled correctly. So I don't see how LLMs could handle these cases correctly in the near future.
Why on Earth would you want to replace that with an LLM that might hallucinate?
I think a system that parses the code and uses verifiable rules to hide invariant changes would have a much easier time being adopted. I may be biased since I work on SemanticDiff (think difftastic but for VS Code and GitHub) and have implemented such rules myself. There have been several times where I've thought about implementing a new rule, only to find out that some obscure edge case would not be handled correctly. So I don't see how LLMs could handle these cases correctly in the near future.
We still see this technology being able to handle edge cases (as you mention) such as certain movements of blocks of code to another file.