News for hackers. Stuff that matters.
Re:Why is so much memory needed for deep neural networks? by dnautics on Thursday March 23, 2017 @04:44AM (#13938195)
I didn't see any mention of striding. For convnets, you have to do a very unwieldy data unroll that either copies your data by (nxn) where n is the width of your kernel, or you have to manually stride, which can cause all sorts of problems with memory bank collisions, paging problems, etc.
Re:Why is so much memory needed for deep neural networks? by jacquesm on Thursday March 23, 2017 @08:10AM (#13939053)
Yes, this is a real problem. I can easily tell if my GPU is having striding issues by looking at the power consumption of the machine. If I manage to get > 40% utilization I consider that pretty good. GPUs are useful but far from perfect for this application and the quickest way to sink performance is to 'starve the beast' by having sub-optimal memory access patterns.
Re:Why is so much memory needed for deep neural networks? by deepnotderp on Thursday March 23, 2017 @04:19PM (#13943858)
So don't use a GPU, don't suffer from old paradigms :)
Re:Why is so much memory needed for deep neural networks? by radarsat1 on Thursday March 23, 2017 @09:34AM (#13939540)
Couldn't you organize the input data in a "pre-strided" layout? Not sure if that makes any sense.. But I mean, re-arrange the pixels so that consecutive pixels per stride are consecutive in memory.
Re:Why is so much memory needed for deep neural networks? by dnautics on Thursday March 23, 2017 @03:54PM (#13943600)
> For convnets, you have to do a very unwieldy data unroll that either copies your data by (nxn) where n is the width of your kernel

Justin does a WAY better job of explaining it than I do

https://www.youtube.com/watch?v=dUTzeP_HTZg&t=36m30s