Re:Large Language Models for Compiler Optimization
by quadrature on Sunday September 17, 2023 @08:16PM
(#37550736)
I wonder if this requires a 7B parameter model.Assembly has a small grammar and is very constrained compared to natural language.
[ Parent
| on HN ]
Re:Large Language Models for Compiler Optimization
by kaba0 on Monday September 18, 2023 @03:54AM
(#37553474)
I don’t see why would the output language’s complexity matter - that’s clearly not the hard part. You need plenty of “thinking” to do for outputting sensible assembly, let alone whole programs.
With that said, it is not doing neural compilation as others have mentioned, it’s only about ordering/enabling different phases of the compiler based on ML, over the current, simpler heuristics.
Re:Large Language Models for Compiler Optimization
by quadrature on Monday September 18, 2023 @10:16AM
(#37556417)
Exactly, since its not doing whole program synthesis im thinking it could be done with fewer parameters. However program synthesis is part of the loss function.
Re:Large Language Models for Compiler Optimization
by cec on Monday September 18, 2023 @05:00PM
(#37561979)
Program synthesis is part of the loss function, which is what makes it a auxiliary learning task.
We haven’t experimented with model size yet, we just used the same configuration as the smallest Code Llama. We did play with dataset size and found thah performance tracks the usual scaling laws. Details in the paper
Re:Large Language Models for Compiler Optimization
by quadrature on Monday September 18, 2023 @11:08PM
(#37565191)
Thanks for the reply!. This is really interesting research, hope to see more from your team.