News for hackers. Stuff that matters.
Re:Program Synthesis with Large Language Models by YeGoblynQueenne on Wednesday August 18, 2021 @09:57AM (#28220976)
Hello Augustus and thanks for posting the link to your co-authored paper on HN.

I will have to read the paper more carefully however having quickly scanned the paper it seems that it only reports empirical resuls. In particular, there seem to be no theoretical results about lernability of programs from natural language specifications using large language models. To make it more plain - how do we know these techniques work as well as reported on problems other than the ones in the dataset introduced in your work?

Note that I'm not asking why you introduced a new dataset, this seems to be motivated in the abstract. I'm asking: how do we know how well this kind of thing works ("this kind of thing" being what it says in the title) in the general case?

There are no comments on this comment yet.