albertan017

We find several work on refining Ghidra decompilation results with GPTs, that could be another interesting directions!

Yes, GPT4 is very impressive, as it's not directly trained on the decompilation. We're working on improving our model, please keep watching updates!

Thanks. We acknowledge that an LLM cannot completely replace human expertise in decompilation, much like GPT-4 has not achieved true human-like intelligence. However, the aim of our llm4decompile project is to do something like GPT-4, and offer assistance and enhance productivity in the decompilation process.

As for test suites, it's one of our project's main challenges—figuring out which functions satisfy the expectations of reverse engineers, how to autonomously produce high-coverage test suites, and how to objectively qualify decompilation outcomes without relying solely on human judgment. Looking forward to your advices!

Ideally, with a substantial dataset of obfuscated JavaScript and corresponding raw code, a language model could potentially make good predictions. The first key difficulty, however, is collecting a large-scale dataset and setting up a system for automatic compilation and segment out the binary-source pairs.

We're interested in the toolchain, could you share the link or reference to it? GPT4 does an amazing work, we're also very surprised that it can work.

Yes, it's not easy to train a 33B model. An interesting point is, naive fine-tuning, which means if one followed the standard way to fine-tune the model. Training a larger model is tricky, not only the data amount matters, everything like data cleaning, learning rate, and decays will affect the final performance.

Thanks! Our initial experiments indicate that for simple cases, such as short snippets (tens of lines) of code without external dependencies, the LLM can decompile very well. However, for more complicated examples, it tends to offer speculative solutions, and the utility of these results is challenging to assess. The determination of whether the decompiled output is correct or useful is subjective and lacks a universal standard. One approach we're considering is utilizing GPT-4 as a benchmark to evaluate other models' performance. We're open to further suggestions to refine our evaluation methods.