The invention of compilers was a huge watershed moment for programming. All the manual labor you had to do, keeping track of memory layouts and registers and stack frames, was handled automatically for you. Yet the result is not that programming became easier - it's that programs became more complex. A journeyman programmer today can write Tetris in a few hours, while it took how many months for the original? But modern games are things like Starcraft 2 and Halo, not Tetris and Space Invaders.
Same with databases. It used to be that you had to think in terms of bytes in sectors on a disk. Now we have filesystems and databases and query languages to abstract that all away from us. But that doesn't mean programming the systems becomes easier, it means that the volume and variety of data stored increases.
It used to be that domains like speech recognition, natural language processing, text mining, computer vision, OCR, etc. were completely off-limits for computational approaches. Now we have standard libraries for them, and standardized tools for generating those standard libraries. But that just means the tools get applied to progressively more problems, not that solving those problems becomes easier.
I think this is what Marc Andreesen meant when he said that "Software is eating the world". There will eventually come a time when no more software engineers will be needed, when they have automated themselves out of a job. But that time will come when everything we do is computerized, when every aspect of modern life is ruled by algorithms and data. Until then, advances in technology just mean computation can be applied to more and more problem domains.
It seems to me, and I may be wrong on this, that the complexity of the problems that most programmers are approaching doesn't scale on a 1:1 basis with the power of their tools.
If all of our "CRUD apps" were truly just CRUD apps we would have built them all in 10 minutes with rails scaffold.
i like your name too!
Greater labor efficiency in the production of a good only decreases the quantity of labor directed to producing that good demanded in the market when there is either some non-labor (or different kind of labor) input to production of the good (e.g., in agricultural productivity, land would be the non-labor input) whose supply is limited, or quantity demanded of the output good goes past the point where cost declines are responded to 1:1 with increased quantity demanded.
Given the number of existing and potential markets for software and the relative transferrability of programmer skills and effort between those marktes, I would be surprised if the latter was true, and I don't see an obvious non-programming constraint on the production of software that would be limiting.
The trend so far has been that when you improve programmer productivity, you increase the value of what we do, which increases the number of things that we can be useful for. This increases demand.
When I started programming in 1973, my boss (who had a PhD in computer science) said that there was a factor of 100 difference in the productivity of different programmers he had worked with.
Even if this differential productivity remains, as tooling gets better I have to wonder how many software engineers will be needed in 20 years if all programmers become much more efficient?