One thing that makes me depressed about ChaGPT is that the software engineering behind it is not some 1996 Doom Carmack style wizardry. It's attention mechanisms, a transformer architecture, and lots and lots and lots of data. The data does all the programming. I look at a lot of AI projects and there's not a lot of code. Sure it takes a lot of work to understand it, but in the end it's not a lot of code.

This makes me think that in the future programming will be just about collecting data and then doing things with the data. We will all be data generators and collectors with a few math PHDs in the middle working on the activation functions and so forth. The rest of us will be stringing models together Stable Diffusion style or writing the billing code.