Re:Practical Deep Learning for Coders
by Datenstrom on Sunday June 09, 2019 @02:26PM
(#20140881)
If you want to take a look at a framework that isn't a terrible mess and that one person can understand both Darknet [1] and Flux [2] are really nice. Learned a lot from reading their source.
Re:Practical Deep Learning for Coders
by monadmancer on Sunday June 09, 2019 @03:07PM
(#20141175)
These are refreshing. Redmon's resume is, the first of its kind I suspect.
Part 2 of this course, which will be released later this month, is a step by step walkthrough of building the FastAI and PyTorch libraries from scratch (minus autodiff).
I recently published a series of 16 blog posts Deep Learning from Scratch to GPU that does exactly that! [0]
I also started working on a book titled "Deep Learning for Programmers"[1] that takes it even further! You can subscribe today and start reading the drafts as they are written.
Re:Practical Deep Learning for Coders
by umanwizard on Sunday June 09, 2019 @02:39PM
(#20140989)
Not a complete answer to what you're looking for, but you might find this dive into PyTorch internals interesting: http://blog.ezyang.com/2019/05/pytorch-internals/
Re:Practical Deep Learning for Coders
by monadmancer on Sunday June 09, 2019 @03:09PM
(#20141185)
This is actually very close to the flavor I had in mind.
Now, I don't know if you're implying that fast.ai specifically should include this in their course, and you're saying this as feedback, but there's tons of resources online for it. If you're looking for a guide i highly recommend this: http://neuralnetworksanddeeplearning.com/chap1.html.
you're talking about thousands on PhD level work in autodiff and optimization and whatever else. you're not going to learn that in one PhD let alone one class.
I think that’s only necessary for a good library. I think you could probably make a bad or simple one with much less.
I've been writing a C-based deep learning library for the past year or so, which I've recently been trying to clean up to be a bit more presentable. It was pretty heavily inspired by Darknet, although it admittedly has far fewer features: https://github.com/siekmanj/sieknet
I think there's also a need for a very low level course in deep learning. I.e. on the level of someone who wishes to write their own deep learning library. Because from high up, sure it all looks like the chain rule, but down low, it gets messy quickly if you want to write a high performance library on your own.