Btw, very tangentially, did you used to work in a startup with a certain zap to its name? I recognise your handle :)
The problem is this is never how the transition would happen.
Today how does it work? Go far enough back. You'll have a user with problem X. Someone translates this problem into requirements to have others implement. You would never take the requirements and give them to a general AI because, as you said, you'd have to ensure it understood them.
What would actually happen is you'll tell your general AI that you need to solve X and it'll figure out the best way to do it and just do it. They wouldn't write software in the sense we have it today. They wouldn't produce an app. They would, instead, solve problems.
Here's an example. Today you think "I'm really hungry. It would be great to have an app that could show me what's around me then have someone pick it up for it after I make a selection". You have some basic requirements in your head, you write some software, you test it out. Maybe your app calls an API to place an order for pizza then calls an Uber to pick up your pizza and then bring it to your current location.
With a general AI you'll feel hungry and the AI will know you're in the mood for pizza from Dave's Pizzeria 3 blocks down. It'll dispatch an automated vehicle to pick up the pizza from a likely automated kitchen and bring it straight to you. You wouldn't even need conscious thought into it if we have a way to perfectly connect the AI and our brain.
Design is so much more important than code, knowing what you want and knowing what you need is more than half the battle. Finally don't over look that an AI will be 24x7 and no person can have that output
I don't think it will write "perfect code" whatever that means. It will write code with its own idiosyncrasies its own writing style, etc.
Suppose such an AI exists, one that can write perfect code and get the job done. How do you tell the AI what to code? Any language specific and accurate enough to tell the AI what to code, is in itself a programming language. Someone needs to write in that language, and that person is a programmer. All the AI does is make for a better compiler.
Programmers will be (and already are) augmented by technology and AI, but someone has to tell the computer what the goal is.