Part of the problem with DS is that there are known engineering solutions without good theoretical explanation, that is, we just brute forced some basic engineering solutions and keep hacking at them to get more performance, but don't have a good theoretical model of why those things do what they do. (Or have a theoretical model too complex to compute useful insight, which is another kind of problem.)

I think that this kind of approach is going to get us pretty far -- see how far we got on building bridges with no more sophisticated insight in to gravity than "things fall down; some things are heavy" or material science than "some things hard and not bend" -- but I think that ultimately, our longer term goals like AGI will require that we build theoretical models capable of explaining our current engineering progress and predict a new, deeper valley of progress to move to.

Of course, that's a lot like saying making progress on physics paradigms requires explaining high temperature superconductors (or other currently open problems). It's actually pretty normal for a field to be a mix of engineers being ahead and theorists being ahead.

Mathematical rigor is really a sign that you're in established territory and explorers have moved on, and much of data science is still under heavy exploration.

There are tons, tons, of examples of "established territory" in CS that do not exhibit "mathematical rigor". When something works in CS people seem to say "good enough" and use it. In mathematics (or as you pointed out, in physics) it becomes an "open question" and people begin the hard work of explaining the phenomenon by applying rigor. In maths, it's a constant march to push the boundaries, where the boundary is defined as that which is interesting but not yet well defined. In CS it seems that the boundary is that which is not yet a solved problem, where "problem" is something that needs getting done. Once, in CS, we can get it done, people move on. It's not often that in the enterprise CS process (academic is a whole other beast) it is assumed that there should be an application of rigor or an attempt to well define "solutions" before moving on. There is simply the accumulation of technical debt. But, in my opinion, technical debt has amassed at the systemic level to such a point where it's starting to look like a ponzi scheme. Sure, it works, so long as we keep throwing good money after bad. But, stop investing, and one can see it for what it is. In maths on the other hand, step away from it all for a year or two, and when you go back everything is still just as valuable and beautiful.

edit: what i would love to see is a "category theory" for programming languages / paradigms such that moving between them is well defined. it boggles my mind that translating between two programming languages isn't trivial. if both are well defined, one should be able to translate one to the other precisely given one is willing to define the translations. there should be zero guess work or heuristics in the process.

It is baffling that machine translation on human language is approaching human accuracy but it is still far behind on programming language, given that it is supposed to be more closed/matching to metal/machine.

I find that the opposite of baffling: human language is information sparse and redundant; computer languages are information dense and, by choice, are made as unredundant as possible. It just seems like trying to hit a big fluffy cloud on one side and a bald tree on the other. Obviously the big fluffy thing is easier than the spindly, narrow one.

How does machine translation do at poetry, in the sense of capturing the figurative meaning and stylistic elements, not just the literal meaning of the tokens?