Re:Neural Networks from Scratch
by Imnimo on Monday October 11, 2021 @03:21PM
(#28831951)
Certainly there's a lot you can do without understanding backprop - you can train pre-made architectures, you can put pre-made layers together to build your own architecture, you can tweak hyperparameters and improve your model's accuracy, and so on. But I also think you will eventually run into a problem that would be much easier to debug if you understand backprop. If your model isn't learning, and your tensorboard graphs show your gradient magnitude is through the roof, it'll be much easier to track that down if you have a strong conceptual model of how gradients are calculated and how they flow backwards through the network.
[ Parent
| on HN ]
I could be wrong but I think it’s possible to understand backprop without being able to apply the chain rule from memory.