Yeah I have found copilot to be lacking when it comes to understanding the structure of certain things. Like when you ask it to add tests, it can figure out where the test files go, but can't seem to insert code into existing tests, so you have to rearrange code fairly often.

I think that code generator models need to understand the abstract syntax tree, like using treesitter to better understand the structure, and as you said, the patterns involved in the code. Plus generators could then check that their new code correctly implements a pattern.

I'd love to be able to feed a repo into a model and get documentation and diagrams out of it.