My understanding is you don't explicitly handle that. For whichever place you're evaluating, you have some computation graph, and you use the elementary operators on that graph. So, if you have defined ReLU with an `if x >= 0 return x` clause, then if you evaluate the derivative at 0, that's the branch you go down, and you say the answer is 1.