Re:How to build your own neural network from scratch in Python
by ScoutOrgo on Friday October 05, 2018 @12:23PM
(#18149388)
This notebook from fast.ai is more in line with what op mentioned. It starts from nothing but Python and slowly swaps out chunks of code with functions/classes from Torch/fastai:
https://github.com/fastai/fastai/tree/master/courses/dl1