Re:Grid: AI platform from the makers of PyTorch Lightning
by tmabraham on Thursday October 08, 2020 @06:05PM
(#24724424)
In the above-linked GitHub issue, the Google TPU team is now giving an ETA of early 2021. At that point PyTorch TPU training (including on TPU pods) should be equivalent to TF TPU training. But I think my point still stands that as long as PyTorch TPU training is faster than GPU training, even in its current state, there's nothing wrong advertising TPU support now.
So you can't blame the PyTorch team. If there's anyone to blame, it's Google Cloud. In the meantime, I don't think there's any harm advertising PyTorch with TPU support if running on TPUs with PyTorch is often much faster than running on GPUs with PyTorch.