Yeah, this is something that even a lot of data science leaders don't understand.

As an example, I recently discovered a bug in a production system that was costing many millions of dollars, that essentially happened because the team was told to go off and implement a shiny new ML model rather than understand and incrementally improve the system.

It's incredibly depressing.