Vulkan/SPIR-V looks promising just needs chip vendors (ARM, Qualcomm, AMD, Intel) to come together and invest in CuDNN equivalents.

Although I reckon deep learning on mobile (at least for some use cases like cameras) will use dedicated silicon from Movidius etc and ultimately be embedded in the camera chips directly