All currently used deep learning algorithms are special cases of neural networks. The reason why this is called "deep" learning is that before 2006, no one knew how to efficiently train neural nets with more than 1 or 2 hidden layers. (Or could, because of computing power.) Thanks to a breakthrough by Dr Hinton, this is now the case.
But all models used are neural nets. It's just that a vast amount new algorithms for training them have been developed in the last years and people came up with new ideas on how to use them.
But it is all neural nets. And that's the whole beauty of it.
These are all neural nets (with some bells and whistles in some cases like tied weights, pooling units, etc) trained exactly as they were trained before using stochastic gradient descent or LBFGS. We did come up with a lot of tricks for making SGD work though, like momentum terms, clamping of weights during learning, dropout, unsupervised pretraining, etc., but in large part it's just a lot more compute power. These networks just turned out to work very well when you have a LOT of (fairly homogeneous) data and can afford to scale them up computationally. And that's pretty awesome, looks like we have a powerful hammer and there are plenty of nails lying around :)
As far as I can tell there hasn't been any single revolutionary breakthrough in this field...we just keep getting more computing power, discovering better tricks and heuristics, and trying to build larger and larger networks.
edit: I'm not sure I was clear enough-- the term "neural network" is a misnomer that encompasses extremely different models that are largely unrelated except for being vaguely inspired by the brain. A vanilla multilayer-perceptron is essentially a generalization of logistic regression. Restricted Boltzmann Machines are different beasts-- they're a restriction of undirected graphical models made amenable to efficient training. Recurrent neural networks aren't in any way a minor extension of other neural networks-- you need different terminology to talk meaningfully about them and they essentially don't have reliable training algorithms. This latter class can be viewed as Turing-equivalent computation, but they're not at all the same as the models in the original article.
Years later, Swiss researchers (Dan Ciresan et al) found that you can train neural nets with backprop, but you need lots of training time and lots of data. You can only achieve this by making use GPUs, otherwise it would take months.
Check out the publications by Ciresan on MNIST, have a look at Hinton's dropout paper or at the Kaggle competition that used deep nets. Or try it yourself and spend a descent amount of time on hyper parameter tuning. :)
It's been awhile since I read the paper, but as I recall it involved training an unsupervised model layer-by-layer (training a layer, freezing the weights, then training another layer on top of it).
The deep learning / RBM tutorial here is quite good and explains the technique.
Deep learning, on the other hand, is about using layers of classifiers to progressively recognize higher-order concepts. In computer vision, for example, the first layer of classifiers may be recognizing things like edges, blocks of color, and other simple concepts, while progressive things may be recognizing things like "arm", "desk", or "cat" from the lower-order concepts.
There's a book I read a while ago that was super-interesting and digs in to how one researcher leveraged knowledge about how the human brain works to develop one of these deep learning methods: "On Intelligence" by Jeff Hawkins (http://www.amazon.com/On-Intelligence-Jeff-Hawkins/dp/B000GQ...)