News for hackers. Stuff that matters.

A comment on Type-constrained code generation with language models.

Re:Type-constrained code generation with language models by informal007 on Wednesday May 14, 2025 @01:19AM (#43981088)
Would it better if we move the feedback loops into RL-stage of LLM training?

Are there some related works?

[ Parent | on HN ]
Re:Type-constrained code generation with language models by nielstron on Wednesday May 14, 2025 @02:08AM (#43981352)
we were thinking about doing exactly this, the closest current work is probably the amazing "Learning Formal Mathematics from Intrinsic Motivation" by Poesia et al (they use constraints too increase the likelihood of generating correct theorems/proofs during RL)

https://arxiv.org/abs/2407.00695

[ Parent | on HN ]
Re:Type-constrained code generation with language models by informal007 on Wednesday May 14, 2025 @03:14AM (#43981746)
Yes, RL works well in fields where answer can be verified in different degree. That's why AlphaGo success, it also should work in code generation and math.
[ Parent | on HN ]
Re:Type-constrained code generation with language models by imtringued on Wednesday May 14, 2025 @06:33AM (#43982899)
Your reward function can simply be the distance between the constrained output and the unconstrained output, that way you won't even need synthetic data, just a dataset of prompts to RL against.
[ Parent | on HN ]
Re:Type-constrained code generation with language models by informal007 on Wednesday May 14, 2025 @11:33PM (#43991578)
How to get "unconstrained output" and evaluate the distance between them?

Evaluation method which can decide distance between two sentences is hard to find, best option is closed-source LLM API even it's not the most ideal option. As a result, we also must use current LLM to improve our models.

[ Parent | on HN ]