This has been my experience for the vast majority of cases when I worked in software quality. I think the Dunning-Kruger effect may be real, but possibly a special case describing a much smaller subset.
I've worked for orgs where software was an enabling product and not the product given to their customers. The leaders took the perspective that they "do not need world-class ingredients to make a world-class product". Meaning, the software didn't have to great for them to create a great end-product. So it's little wonder why there was little motivation to improve software quality. I think if properly motivated the competence would follow.
>I can say that a not-insignificant amount of projects in corporate America (and probably other countries too, I just have not worked outside of the US) really are "busy work", that create little or no actual value
You may be interested in the book Bullshit Jobs by the late David Graeber
It's really hard for some people (myself included) to give good, timely work when they simply do not care about the stuff they're working on. It's entirely possible that Pat is actually a very good programmer on projects that he cares about (following code standards, meeting deadlines, "going above and beyond" etc), but may have been put on a project that he doesn't feel is interesting or mentally stimulating.
From experience, I can say that a not-insignificant amount of projects in corporate America (and probably other countries too, I just have not worked outside of the US) really are "busy work", that create little or no actual value to the company or the world, are not intellectually interesting, and are largely worked on to justify the existence of some middle-manager. When I am working on a project like this, especially when it's the only thing I'm assigned, it's incredibly hard for me to care about the results of it.
Granted, I might be a crappy programmer, but I think the dichotomy between "good programmer" and "bad programmer" is needlessly reductive.