For Deep Learning, start with MNIST. The 1998 paper[1] describing LeNet goes into a lot more detail than more recent papers. Also there's an excellent video from Martin Gorner at Google that describes a range of neural networks for MNIST[2]. The source code used in his talk is excellent[3].

[1] http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf

[2] https://www.youtube.com/watch?v=vq2nnJ4g6N0

[3] https://github.com/martin-gorner/tensorflow-mnist-tutorial

Starting with DL? That's like learning calculus before geometry... YMMV?

Convolutional nets for digit recognition are certainly easier to learn than ML generally, but I wasn't suggesting to start with Deep Learning. I was suggesting that when studying Deep Learning, to start with MNIST.

It was easier for me to start with the LeCun 1998 paper than to watch all the theorem-proving in the online courses, but that's just personal preference.

Sure, but I take it the original comment wasn't exactly by someone with some ML background. And getting to grips with log likelihoods, (cross-) entropy, linear/logistic regression, evaluation metrics, and maybe even some Bayesian statistics might be rather helpful before jumping on the DL bandwagon.

While there are far too many hardcore statisticians and academics who love their theorems more than anything, not all classes are that way. I think I'd have loved it if I could have learned ML from today's MOOCs, instead of those theorem provers and formula speakers I had to deal with (and pass real-life exams you can't repeat every 8 hrs...)

I don't have an ML background and I had no problem understanding the LeCun 1998 paper. Naturally, the more ML one knows the better, I'm just encouraging people to dive in and try without getting intimidated.

Anecdotally, one astonishing observation I often make is that "breakthrough" papers [1] are nearly universally among the most accessible, clear and easy to follow. From Watson and Crick on DNA in MolBio, to Backpropagation by Hinton in ML, to Cox' survival model in Statistics, the most significant advances often tend to be the "easiest" to understand (in hindsight only, naturally).

[1] http://www.nature.com/news/the-top-100-papers-1.16224

thanks! definitely keep this for when I'm starting to get to DL!

Without doubt do the Andrew Ng course on Machine Learning.

https://www.coursera.org/learn/machine-learning

It's excellent.

I've started it already, seems like this one and the AI udacity course are the two best ones for now :) Thanks!

Another resource I've found really useful is course.fast.ai - it presents a very practical approach to deep learning in a way that would be particularly familiar to someone who has done any amount of programming.

I had taken the Ng course, which I recommend to get a basic understanding of ideas, but, the fast.ai course enables you to compete in Kaggle competitions in fairly short time.

Just watch the course.fast.ai intro/overview video, and decide from there.

This is a very well-written intro to Deep Learning: http://neuralnetworksanddeeplearning.com

I've been going through the online book. I've been enjoying the tutorials so far.

True, it is not so easy to figure out. I faced similar situation but I watched Udacity ML course and made an attempt.

Read my post:

"How I wrote my first Machine Learning program in 3 days"

http://blog.adnansiddiqi.me/how-i-wrote-my-first-machine-lea...

I think this is a great resource

https://www.r-bloggers.com/in-depth-introduction-to-machine-...

It doesn't cover deep learning at all, but contains material about many other ML techniques.