The problem with using alive2 to verify LLM based compilation is that alive2 isn't really designed for that. It's an amazing tool for catching correctness issues in LLVM, but it's expensive to run and will time out reasonably often, especially on cases involving floating point. It's explicitly designed to minimize the rate of false-positive correctness issues to serve the primary purpose of alerting compiler developers to correctness issues that need to be fixed.
Additionally, they train approximately half on assembly and half on LLVM-IR. They don't talk much about how they generate the dataset other than that they generated it from the CodeLlama dataset, but I would guess they compile as much code as they can into LLVM-IR and then just lower that into assembly, leaving gcc out of the loop completely for the vast majority of the compiler specific training.
It seems like somehow build systems were invoked given the different targets present in the final version?
Was it mostly C/C++ (if so, how did you resolve missing includes/build flags), or something else?
> (A <=> B) < 0 is true if A < B
> (A <=> B) > 0 is true if A > B
> (A <=> B) == 0 is true if A and B are equal/equivalent.
TIL of the spaceship operator. Was this added as an april fools?
It's useful for stable-sorting collections with a single test. Also, overloading <=> for a type, gives all comparison operators "for free": ==, !=, <, <=, >=, >
In C++20 the compiler will automatically use the spaceship operator to implement other comparisons if it is available, so it's a significant convenience.
What they missed is to mention verification (they probably don't know about alive2) and comparison with other compilers. It is very likely that LLM Compiler "learned" from GCC and with huge computational effort simply generates what GCC can do out of the box.