Yep! No GCC on this one.
And yep, that's not far off how the pretraining data was gathered - but with random optimisations to give it a bit of variety.
Re:Meta Releases LLM Compiler
by boomanaiden154 on Friday June 28, 2024 @06:38PM
(#40826071)
Do you have more information on how the dataset was constructed?
It seems like somehow build systems were invoked given the different targets present in the final version?
Was it mostly C/C++ (if so, how did you resolve missing includes/build flags), or something else?
We plan to have a peer reviewed version of the paper where we will probably have more details on that. Otherwise we can't give anymore details than in the paper or post, etc. without going through legal which takes ages.
Science is getting harder to do :-(
Additionally, they train approximately half on assembly and half on LLVM-IR. They don't talk much about how they generate the dataset other than that they generated it from the CodeLlama dataset, but I would guess they compile as much code as they can into LLVM-IR and then just lower that into assembly, leaving gcc out of the loop completely for the vast majority of the compiler specific training.