You mention searching the tree of potential programs that satisfy a set of constraints—but what's the fastest way to search that tree, both in making edits that are contextually sensible, and in using prior programs to factor out common patterns?

If you want to go off the deep end of the possibilities of NNs in compilers, I suggest you look up wake-sleep program synthesis at the least: https://dl.acm.org/doi/10.1145/3453483.3454080