Yes, after we create something from scratch, we then see how to use implementations from libraries such as PyTorch. For basic operations like convolution, whilst we do create a GPU-accelerated version, it's not as fast as PyTorch/cudnn's highly-optimised version, so we use the PyTorch implementation for the rest of the course, rather than our handmade version.

So that means that nothing is mysterious, since we know how it's all made, but we also see where and how to use existing libraries as appropriate.

In practice, during the course we end up using PyTorch for stuff like gradients, matrix multiplication, and convolutions, and our own implementations for a lot of the stuff that's at a higher level than that (e.g. we use our own ResNet, U-net, etc.)