Yes, they have been trying to create 'sufficiently formal human readable text' to spec out projects; not detailed enough to execute by a computer but formal and precise enough so humans know exactly what they are getting. That still doesn't work at all and that is between humans. If the specs are clear enough, the act of programming is already mostly not the issue, however, they never are. I am looking forward to ML helping me writing boring code (which CoPilot already does, but again, that's not really where time/energy is spent anyway) and protect against security issues, scalability issues and all kinds of bugs (it could rewrite algo's it knows; it could recommend libraries that I should use instead of the crap I rolled myself etc).

Fully automating software engineering won’t happen until AGI. As a good Yuddite I expect us to have bigger problems when that happens.

You need an agent with a large and coherent world model, in order to understand how your programs relate to the real world, in order to solve business tasks.

This isn’t something any program synthesis tech currently available can do, because none of it has a coherent world model.

GPT-3 comes closest to this, but isn’t able to engage in any kind of planning or abstract modeling, beyond semi coherent extrapolations from training data.

Maybe scaling up GPT by a few more orders of magnitude would work, by generating an emergent world model along the way.

What is a "Yuddite?" I tried Googling for it and got the impression it was LessWrong forum terminology for people who believed too strongly in LessWrong, but I couldn't find many references.

I believe he's referring to "luddites" -- a group of people who resisted technological innovation during the industrial revolution.

Luddite but mixed with "Eliezer Yudkowsky" who is a researcher working on the problem of friendly AI (or whatever they're calling it these days). Basically trying to prevent skynet.

The GP is saying that once we have AGI, then "AGI is going to make the human race irrelevant" outweighs "AGI makes software devs irrelevant".

That’s the idea.

I am a follower of Elizier Yudkowsky.

I would actually argue the programmers job has never been 100% writing the code, it’s always been interpreting, fixing and decoding the ideas of others.

The older I get the more I see it has not been about programming for most tasks for quite a long time. In the early 80s it was a bit more (but not even much more); at that time as well I spent most of my time debugging and changing behaviour slightly (but in a lot of pages) instead of just cranking out huge bags of code.

I would argue that we figured this out over 50 years ago but oddly enough some people still hold onto the idea.