This. Thinking about any of the sizeable apps I develop and maintain, 90% of the job is just thinking of edge cases in the business logic as transmitted to me by the people running the business I'm writing an app for. About 10% is actually coding and testing. While I'm impressed with copilot/GPT's ability to write SQL queries and imitate my coding style to write functions that usually have no obvious bugs, it's the subtle, non-obvious bugs that I'm worried about. Anything it writes demands careful reading and correction - just like anything written by ChatGPT demands a great deal of fact-checking. So much so that it's not really efficient. The idea of reviewing a whole code branch it's written fills me with dread, because who knows where the corner cases are lurking? And when they arise, who knows what else patching them might affect? At least with my own code or code written by humans, I know where things are and can deduce why they were written that way. With copilot, there's no why.

In a sense it's a metaphor for the whole problem of LLMs ... or AI in general. When unlimited compute lets you replace logical deduction with brute force heuristic reduction, it makes it impossible to trace routes and find bugs.