>The limitations to GPT3 codex and AlphaCode seems to be they're relatively weak at selection

This really does seem like the key here--the knowledge apparently is all in the language model, we just haven't found the best ways to extract that knowledge in a consistent and coherent manner. Right now it's just: generate a bunch of examples and cherry pick the good ones.