Exactly right now, I am helping a big oil and gas company have a process simulation software to correctly converge on a big simulation. Full access to the source code, need to improve the Newton method in use with the right line search, validate the derivatives, etc.
I do think that for most of the people, you are right, you do not need to know a lot, but my philosophy was to always understand how the tool you use work (one level deeper), but now the tool is creating a new tool. How do you understand the tool which has been created by your Agent/AI tool?
I find this problem interesting, this is new to me and I will happily look at how our society and the engineering community evolve with these new capacities.
It might be as simple as creating awareness about how everything works underneath and creating graduates that understand how these things should work in a similar vein.