What you're missing is that the gigantic initial success models have consistently gotten much much lighter after the hype cycle moves on from the initial release. In my own domain, WaveNet - speech synthesis vastly outperforming previous methods - went from fantastically expensive to being able to run on cheap phones. You can run bird song id on your phone in real time in the Merlin app. There's a proliferation of light neutral networks for lots of specific use cases, and it's already happening all around you.