Re:Why is so much memory needed for deep neural networks?
by dnautics on Thursday March 23, 2017 @04:44AM
(#13938195)
I didn't see any mention of striding. For convnets, you have to do a very unwieldy data unroll that either copies your data by (nxn) where n is the width of your kernel, or you have to manually stride, which can cause all sorts of problems with memory bank collisions, paging problems, etc.
[ Parent
| on HN ]
Re:Why is so much memory needed for deep neural networks?
by jacquesm on Thursday March 23, 2017 @08:10AM
(#13939053)
Yes, this is a real problem. I can easily tell if my GPU is having striding issues by looking at the power consumption of the machine. If I manage to get > 40% utilization I consider that pretty good. GPUs are useful but far from perfect for this application and the quickest way to sink performance is to 'starve the beast' by having sub-optimal memory access patterns.
Re:Why is so much memory needed for deep neural networks?
by deepnotderp on Thursday March 23, 2017 @04:19PM
(#13943858)
So don't use a GPU, don't suffer from old paradigms :)
Re:Why is so much memory needed for deep neural networks?
by radarsat1 on Thursday March 23, 2017 @09:34AM
(#13939540)
Couldn't you organize the input data in a "pre-strided" layout? Not sure if that makes any sense.. But I mean, re-arrange the pixels so that consecutive pixels per stride are consecutive in memory.
Re:Why is so much memory needed for deep neural networks?
by dnautics on Thursday March 23, 2017 @03:54PM
(#13943600)
> For convnets, you have to do a very unwieldy data unroll that either copies your data by (nxn) where n is the width of your kernel
Justin does a WAY better job of explaining it than I do