The whole advantage of deep learning comes from the large number of parameters that can fit very complex feature distributions. If you don't have a lot of data and try to train a deep net on it, you're just going to overfit.

Of course there's the opportunity for transfer learning where you try to adjust a pretrained model slightly for a different task, but that only works if the model already mostly solves your problem. (e.g. it picks up on the necessary features, but the class you want is not one of the possible outputs.)

If your data is unlike any popular task with large datasets available, (say because you're trying to predict something from a small in-house database), transfer learning can't work (because there's no knowledge you could transfer) and you are better off with simpler models.