Why is so much memory needed for deep neural networks?

Re:Why is so much memory needed for deep neural networks? by p1esk on Thursday March 23, 2017 @05:31AM (#13938365)

A brain learns one example at a time, which probably means a gradient descent is not a very good learning mechanism.

[ Parent | on HN ]

Re:Why is so much memory needed for deep neural networks? by jacquesm on Thursday March 23, 2017 @08:08AM (#13939039)

These are not brains, they are not even models of brains even if they use the word 'neuron'.

[ Parent | on HN ]

Re:Why is so much memory needed for deep neural networks? by p1esk on Thursday March 23, 2017 @11:39AM (#13940710)

That was kind of my point. Brain learns well using one example at a time. ANNs don't. Hence my conclusion.

[ Parent | on HN ]

Re:Why is so much memory needed for deep neural networks? by akyu on Thursday March 23, 2017 @12:18PM (#13941175)

You can use gradient descent one example at a time. It still works just fine. The gradients are more unstable, but you will still converge eventually.

[ Parent | on HN ]

Re:Why is so much memory needed for deep neural networks? by p1esk on Thursday March 23, 2017 @01:17PM (#13941785)

"works just fine" is a relative term. I'm pretty sure when I'm learning a (new) alphabet, I don't need to see 1000 examples of each letter 100 times.

[ Parent | on HN ]