News for hackers. Stuff that matters.
Re:Why is so much memory needed for deep neural networks? by femto on Wednesday March 22, 2017 @09:14PM (#13936678)
> Combining memory and processing resources in a single device has huge potential to increase the performance and efficiency of DNNs as well as others forms of machine learning systems.

Isn't that describing a Field Programmable Gate Array: memory embedded in a sea of logic cells?

What's the best resource to get someone who knows FPGAs, a decent amount of maths and 1990s style neural nets up to speed on today's deep neural networks?

Re:Why is so much memory needed for deep neural networks? by jacquesm on Wednesday March 22, 2017 @09:19PM (#13936704)
Re:Why is so much memory needed for deep neural networks? by femto on Wednesday March 22, 2017 @09:44PM (#13936807)
Thanks for prompting me to do further research on this. I didn't realise from the initial posting, that this course is based on prerecorded videos and can be completed asynchronously, to fit in with life. The site say students "need access to a computer with an Nvidia GPU", but I reckon it could be an exceedingly interesting exercise to try and do the course with an FPGA in place of the GPU.
Re:Why is so much memory needed for deep neural networks? by jacquesm on Wednesday March 22, 2017 @09:51PM (#13936842)
Alternatively you could use some GPU cloud instances but the end result of that is probably that it ends up costing you more than it would to buy a low-end GPU that you can complete the course with.

It'd be very interesting to see how you do this with an FPGA, that would be quite the accomplishment, especially given the monstrous compute capability that GPUs give, and the communications between high level (say: python) and low level (a GPU or some other co-processor) have all been worked out and debugged for GPUs.

If there is room for FPGA work in there it would probably be some specialized function.

Regardless I'd love a write-up if you do this.

Re:Why is so much memory needed for deep neural networks? by dnautics on Thursday March 23, 2017 @04:21AM (#13938114)
If my understanding is correct, FPGAs aren't that fast, and while the really small ones are cheap, they don't fit that much. I synthesized a 32-bit multiplier on a spartan 6, and it took up about 1/5 of the footprint. I figure maybe you could fit 4 16-bit fused-multiply adds (which ought to be able to do machine learning) onto a spartan 6, which is not that great.

What is really going to slaughter you for ML performance on FPGAs is memory bandwidth.

If you have access to a really big FPGAs with lots of memory (on the order of $10-30k...) it might make a difference.

Re:Why is so much memory needed for deep neural networks? by jacquesm on Thursday March 23, 2017 @08:00AM (#13938990)
I'd see more applications on the evaluation than on the training side but even in training there might be a few situations where it could pay off, you'd still be stuck with a huge communications issue to solve.
Re:Why is so much memory needed for deep neural networks? by zebrafish on Thursday March 23, 2017 @10:36AM (#13940082)
http://neuromorphic.eecs.utk.edu/pdfs/2016-IJCNN-Schuman.pdf

These include SNN implementations on field-programmable gate arrays (FPGAs), one that is trained using a mixture of unsupervised/supervised learning [61] and another that is trained using a GA [62]

An initial implementation of DANNA on field programmable gate arrays (FPGAs)

Re:Why is so much memory needed for deep neural networks? by posterboy on Thursday March 23, 2017 @01:44AM (#13937619)
> Isn't that describing a Field Programmable Gate Array

hard to say, my wristwatch has processing and memory, too.