But then at the same time the code of an industrial system typically isn't the bottleneck in development. Selecting the hardware and waiting for the physical installation is a far bigger bottleneck on a project than what I do. As a result, in the end it isn't the "Correctness" of code in the sense of it is doing what it was instructed to do properly, but the correctness of matching software to hardware and that's where I've seen several failures in AI. If I tell it I have a Schneider Altavar VFD and I get code to interface with an Allen Bradley PowerFlex that's one issue. But if I get code to interface with a Altavar that expects specific parameter adjustments on the drive that haven't been set that's a separate issue and more typical for the faults I've seen within AI. If 99% of drives are configured to operate pumps and that's what AI learns for training, the moment I need it for a mill, aeration fan, or conveyor belt (which have distinct physical requirements) then we have the issues I've been seeing and it takes a trained eye to spot those deficiencies in my experience.
So it sounds like the main problem is the under-specified hardware. I can definitely see how that would limit the usefulness of LLMs.
I used to work on consoles, including the N64, which are fairly low-level compared with a typical Windows, Linux, or Mac application. But you’re obviously working much closer to the hardware, which is a completely different world altogether.
My experience with C and C++ has also been excellent. I work on very large, high-performance C++ systems used to run airlines and airports, and LLMs have dramatically improved both the quality and the performance of our code.
That said, LLMs are tools, and like any tool they take skill to use well. It took me months of experimenting and learning from others to get where I am now.