There is also this: Mixture of experts. A massive neural net from Google, with multiple submodules that are activated depending on input.

https://medium.com/@thoszymkowiak/google-brains-new-super-fa...