Really? The gradients computed by AD are the exact answer to the following question: if I were to change this input or parameter an infinitesimal amount, how much would it change the output of my function? That is always meaningful (when it is defined), and means what I just said. You can easily make functions where it is not defined, of course, just like you can make a sphere into two spheres with the Banach-Tarski theorem!
But there are vast, vast forests of numerical computation employed in industry, science, finance, engineering, where it is almost always defined.
And even for more “chunky” computations where the non-differentiability is more severe, there are algorithms like REINFORCE that you can use to estimate gradients through these parts.
For example, take this code.
x,y
for (int i=0; i<x; i++)
y += 1
return y
I believe REINFORCE is not of much help either - it's not magic. I'm not aware of any stochastic gradient estimators that are helpful in this case (although if there is a method I'd like to hear about it).
It's true that once you have control flow, the gradient quickly becomes meaningless. I posted an example here: https://news.ycombinator.com/item?id=20892287
That's also the biggest reason I tend to find much of this "differentiable programming" stuff to be overhyped. It's hard to reformulate programs in a way s.t. the derivative can mean something meaningful. And I'm not convinced traditional languages will benefit.
That's not to say that there isn't cases where your program can be formulated to have a meaningful derivative.
See this differentiable ray tracer: https://people.csail.mit.edu/tzumao/diffrt/