The thing is it is not simply the training but the inference aspect would have require an incredible amount of compute compared to the old way of doing it.

Hope Google will do a paper like they did with the Gen 1 TPUs. Would love to see the difference in terms of joules per word spoke.