Yes I used Keras before PyTorch came out. IMO, Google needs to start over rather than try extensive wart removal on something that consists almost entirely of warts. These days Keras has subpar multi-GPU support (two GPUs are often slower than one), less flexibility, and more un-debuggable magic than PyTorch. And it doesn't look like things will be improving much (if at all) with TF 2.0.

And the TPU is a story of its own when it comes to usability. Take a look at their docs. TPU actually has to run on a separate machine, which you need to provision manually using their `ctpu` tool, and training of a trivial network on CIFAR looks like this: https://github.com/tensorflow/tpu/blob/master/models/experim...

Err - i don't get it. What alternative r u suggesting for development which is aimed at eventual production?

I am a complete noob, so i just did Keras (to TF) and GPUs.

If i switch to TPU - its still the same code.

When i get the budget to cluster, i think I am better off with one of the Deepmind libs which TF based.

No?

If you're just using off-the-GitHub models and pressing them into service, sure, Keras is adequate. If your work actually requires research and experimentation (as most practical problems on which actual cash money can be made do) you'll be much better served with PyTorch. The main cost in all this is not actually implementing or training your final model. That can be done in any framework. The cost is finding a model that works really well for a particular task. You will get there a lot sooner and with a lot fewer headaches if you use PyTorch for this more "researchy" part.

As to deployment, once you figure out what works, there's also libtorch, or if that doesn't work for you for whatever reason, implementing a model you know working hyperparameters for can be done in a day or two on whatever framework works for your backend, including TF.