Is running TensorFlow on GPUs still relevant now that google has their TPUs? Other dedicated hardware is also coming.

I don't really follow the ML space, but when you have a good GPU in your PC (which I assume quite a few people here have) you can learn/use Tensorflow by running things locally instead of paying for Google Cloud etc. Why pay extra money if you already have a good GPU in your computer (unless you really need to decrease the time it takes to run the computation etc.)

yes. GPUs end up being far more flexible and the latest generation data center GPUs (Volta V100) have dedicated tensor cores. Also, not everyone can (or wants to) use Google Cloud.

Yes. It will always remaining relevant. Google wants a continued supply of GPGPUs from anyone who will sell it to them. They will take TPU to the max, but they are not in it to compete. They are in it to protect themselves first and to have leverage.