that was one of the reasons for TPUs- they are far cheaper (TCO) than large numbers of GPUs if you're doing massive amounts of training.