The challenge isnt writing code. It is writing code that is relatively free of defects. It falls down hard on that point.

Sounds similar to the arguments against self-driving: "Self-driving itself isn't the challenge. It is self-driving better than humans". AFAIK this is already possible at least under certain conditions or at certain roads.

Do you believe we're at the peak of what will be capable?

As shown by this post, we're still discovering what's possible with the tools we have now.

Also, human programmers aren't amazing at writing defect free code.

Free of defects is the least of the problems. Doing what a customer wants is the major one. One could argue that customers will be able to write their code alone but they'll always have something else to do and will pay somebody to do it, exactly as for all the other parts of their business. On the other side maybe one wouldn't have to know one of the current programming languages, much like we don't have to know machine code nowadays. English and a good dose of perseverance could be enough.

Yeah I now routinely ask it for boilerplate to get things done way faster and it's REALLY good at flawlessly translating from one programming language to another. It'll be a shame when it goes behind a paywall.

On that thought, does anyone even still use Copilot?

I didn't poke any holes into it. I just entered my code and asked it to do some work for me and it simply failed. The problem was a simple 50 line function I copy pasted from stackoverflow. I asked it to do something really easy.

The AI can output a lot of text but can you input a 100000 line code base into it? No you can't. You can't even input 50 lines of code that is already in the data set!

And by failure I mean somethin akin to a blue screen and not that the output was wrong, there was no output!

I wonder though could you have it create a list of comments about the function. then use the comments to generate smaller functions and then generate test data for those functions then build unit test for those functions then have it build something to solve the original problem with the smaller functions along with test data.

You don't even have to do that last step and you'll be doing more debugging than is necessary to do that. Just do the elephant carpaccio yourself then feed it the slices and glue the shit it excretes out together and boom you have MVP.