Beginning deep learning with 500 lines of Julia

Re:Beginning deep learning with 500 lines of Julia by lohengramm on Saturday February 28, 2015 @03:35PM (#9124948)

What the "deep" means in deep learning? Is backpropagation deep learning? Why not just machine learning? I became lost since this term started to get used a lot recently. I have implemented and used a feed forward neural net trained with the BP algorithm once and learned it just as a "machine learning" technique, no deep.

I haven't read this yet though, maybe it explains.

[ Parent | on HN ]

Re:Beginning deep learning with 500 lines of Julia by noelwelsh on Saturday February 28, 2015 @03:46PM (#9124988)

Deep usually means more than one hidden layer. The basic algorithms are similar to back propagation, being based on gradient descent, but there are a lot of tricks (or refinements, depending on your point-of-view) to make the learning more robust and efficient.

Deep learning is a specific area within machine learning.

[ Parent | on HN ]

Re:Beginning deep learning with 500 lines of Julia by sgt101 on Saturday February 28, 2015 @03:58PM (#9125032)

I tried training a 4 layer network in as a young man (because, why not) a day later, no convergence, some calculations revealed that an expectation of many 1000's of days (I have a memory of 1 million, but that could be bs) before the network would converge.

So "deep" networks have been around for many decades, and they haven't, because you couldn't train them. Now we have computers that are 10,000* faster (at least) and training algorithms that are much faster too these architectures are interesting.

[ Parent | on HN ]

Re:Beginning deep learning with 500 lines of Julia by noelwelsh on Saturday February 28, 2015 @04:20PM (#9125100)

This comment nails it. I think it is under-appreciated how much machine learning progress is enabled by increasing computing power. This is not to deny algorithmic improvements, but it's hard to refine algorithms if you can't actually run them.

[ Parent | on HN ]

Re:Beginning deep learning with 500 lines of Julia by lohengramm on Sunday March 01, 2015 @12:57PM (#9128089)

Thank you all above for the explanations.

[ Parent | on HN ]