What about "experts"? Like, I would say there are people out there who are valuable because they know how to do stuff. It's not required for them to explain how they do it, and often times it's the case that they can't, otherwise they would simply explain and we'd all be experts. Rather, we need them precisely because the results they deliver are not able to be broken down into a sequence of steps that anyone could follow - since many of them can't explain. So it's something that "works" but isn't math.

I'll add that eventually experts are replaced, but then by that time there are new experts. The problem domain evolves and what used to require experts is replaced with math, and the new experts are working in the area where things can't be math.

Conceptually I think I'm on point for this, but I don't know if my examples are super good. I'd say business, human language, politics, medicine, and art are all examples of things that have experts. In each of these fields that are things that work, but it's not yet backed up by math.

Maybe it's more accurate to say, given an infinite amount of time and intelligence, everything becomes math? And I think that makes sense, but I'm sort of inclined to believing in an objective, yet logistically intractable reality.