News for hackers. Stuff that matters.
Re:Why is so much memory needed for deep neural networks? by radarsat1 on Thursday March 23, 2017 @10:10AM (#13939854)
> So a transformation called 'lowering' is used to convert those convolutions into matrix-matrix multiplications (GEMMs) which GPUs can execute efficiently.

This confuses me a bit, I'd love to know the details of this "lowering"? How is convolution not already a very straight-forward linear algebra operation?

Re:Why is so much memory needed for deep neural networks? by zebrafish on Thursday March 23, 2017 @10:19AM (#13939940)
https://cs.brown.edu/~sk/Publications/Papers/Published/bck-l...

FrTime induces construction of the dataflow graph by redefining operations through an implicit lifting transformation. Lifting takes a function that operates on constant values and produces a new function that performs the same operation on time-varying values. Each time the program applies a lifted function to time-varying arguments, it builds a new node and connects it to the nodes representing the arguments...

...Unfortunately, this implicit graph construction can be very inefficient. Every application of a lifted function may create a new dataflow node, whose construction and maintenance consume significant amounts of time and space....

...The technique works by collapsing regions of the dataflow graph into individual nodes. This moves computation from the dataflow model back to traditional call-by-value, which the runtime system executes much more efficiently. Because this technique undoes the process of lifting, we call it lowering.

edit* or maybe?:

http://physics.gmu.edu/~joe/PHYS428/Topic5.pdf

slide 16?

Re:Why is so much memory needed for deep neural networks? by radarsat1 on Thursday March 23, 2017 @10:23AM (#13939971)
Interesting. Thanks!
Re:Why is so much memory needed for deep neural networks? by dnautics on Thursday March 23, 2017 @04:04PM (#13943710)
I'm not 100% sure but I think justin is explaining the lowering transformation here:

https://www.youtube.com/watch?v=dUTzeP_HTZg&t=36m30s