Re:Alice's adventures in a differentiable wonderland
by barrenko on Wednesday May 01, 2024 @10:01AM
(#40223373)
Thank you for the summary, and thanks to the OP/author of the book.
I started self-studying programming some time ago, then pivoted to AI/ML and (understandably) ended up mostly studying math, these resources are a boon to my folk.
Quickly skimming the draft pdf at https://arxiv.org/pdf/2404.17625 I can grok it instantly, because it's written in familiar academic language instead of gobbledygook. Anyone with an undergrad math education in engineering, computer science, etc or a self-taught equivalent understanding of differential equations should be able to read it easily. It does a really good job of connecting esoteric terms like tensors with arrays, gradients with partial derivatives, Jacobians with gradients and backpropagation with gradient descent in forward/reverse mode automatic differentiation. Which helps the reader to grasp the fundamentals instead of being distracted by the implementation details of TensorFlow, CUDA, etc. Some notable excerpts:
Introduction (page 4):
Chapter 2.2 Gradients and Jacobians (page 23): Chapter 4.1.5 Some computational considerations (page 59): Chapter 6 Automatic differentiation (page 87): Edit: I changed Chapter 2.2.3 Jacobians (page 27) to Chapter 2.2 Gradients and Jacobians (page 23) for better context.