You can use the approach we used to win the CIFAR 10 training cost section of the competition. If you use fastai/pytorch, then it's ~5 lines of code. Check out lesson 1 of http://course.fast.ai for the basic approach, but when calling `fit()`, add the param `use_clr_beta=(20,20,0.95,0.85)` which will enable 1cycle, and should allow of super convergence. Then train with SGD with a really high learning rate (somewhere from 1-3, generally).