Yes, however most ML engineers don't get to work at Jeff Dean's level to actually do such interesting work. There are very few companies willing to write their own Horovod or distributed PyTorch.