SVMs may still be useful as a classifier sitting on top of a deep model. Plugging the features learned during unsupervised pre-training into an SVM instead of a neural net is perfectly valid.

Lower layers of the "deep" model are (typically) performing non-linear dimensionality reduction. i.e. generating a set of high-level features which make subsequent classification easier than it would have been on the raw input data.