News for hackers. Stuff that matters.
Re:Backpropagation is a leaky abstraction by p1esk on Tuesday December 20, 2016 @04:45AM (#13218586)
Yes, after implementing a simple neural network in C (with AVX, and pthreads), "raw numpy" does sound funny!

On the other hand, try implementing a convnet in numpy, especially the backprop, and you might start feeling some of its "rawness" :-)

Re:Backpropagation is a leaky abstraction by TPCrow on Tuesday December 20, 2016 @09:53AM (#13220262)
As someone who also hand-coded a neural network implementation, forward and back prop as well as an RNN in C, yea "raw numpy" is a joke.

Something I've always hated about people that say "why do I have to write a backprop when TF does it for me?"

Here's why: you go to a company, they want you to incorporate machine learning into their c++ engine. Have fun using numpy, you said you knew machine learning right? implement backprop for me, you can do that right?

Re:Backpropagation is a leaky abstraction by chestervonwinch on Tuesday December 20, 2016 @06:48AM (#13219031)
I know what you mean. To take partial derivatives with respect to the filter parameters from a correlation (or convolution), it's simpler to go down to the component level. However, it's hard to get back up to the matrix/vector level after doing so (to write the operations in NumPy).

I'm developing a model (not exactly a convnet) that uses a correlation step. Because of the above problem and its resulting pure-python loops, I may have to cythonize or use the NumPy C API for the gradient evaluation. Do you know of any examples I could check out that implement partial derivatives w.r.t. a correlation (or convolution) in "raw numpy"?