someone quoted from their intro - they claim this is for grad students.
again, i'm on point and right no matter how much hn plays devil's advocate for things it (hn) has a fetish (overly mathematical treatments of tech).
More seriously, it's about doing the impossible. Formally, some functions are nondifferentiable, period. But it would be cool if we could actually "more or less" differentiate them. For that we'll necessarily need a bag of tricks which is now coalescing into "techniques" and "principles".
Cf. numerical analysis. It takes a page or two to set up your definitions and show that many functions are badly conditioned, period. And yet we still want to compute them, so we've been building the bag of tricks for almost a century now.
Almost the entire document is undergrad numerical topics (finite difference methods, maximum likelihood, Jacobins, hessians, newtons method, etc). This is all well covered material that is soundly not research.
I've been trying to learn about applying gradient descent to a non-neural network problem, following a paper, and have found it very difficult to find introductory resources or code libraries that aren't explicitly geared toward training neural networks and running inference on them.
That’s how most subfields are born. Assorted papers -> monograph -> textbook. The first arrow is defining the subfield as a discrete topic, which is immensely valuable. Only after you have that you can start optimizing for presentation to nonexperts.