News for hackers. Stuff that matters.
Re:Cloud TPUs in Beta by whataretensors on Monday February 12, 2018 @12:34PM (#16360238)
I don't like it. Google is mixing too many things. No way to buy a TPU. No competition from other cloud providers. Proprietary hardware and vendor lock-in.
Re:Cloud TPUs in Beta by puzzle on Monday February 12, 2018 @01:02PM (#16360455)
This is really Tensorflow as a service. You get an IP address and a port you send gRPC requests to:

https://github.com/tensorflow/tpu/blob/master/tools/diagnost...

Presumably, there's a whole server behind that address that has all the right drivers and libraries: details you don't need to care about.

The only partial lock-in is that not all ops are supported and you need to figure if there are any parts of the graph in the critical part that will run on the CPU instead. There's a tool for that:

https://cloud.google.com/tpu/docs/cloud-tpu-tools#tpu_compat...

Competitors could launch something similar that uses GPUs tomorrow. Now, if you don't already use TF and don't want to switch, that's another story.

Re:Cloud TPUs in Beta by whataretensors on Monday February 12, 2018 @01:17PM (#16360592)
That's my point. Competitors are largely moated out by high costs of TPU production and proprietary drivers.
Re:Cloud TPUs in Beta by dgacmu on Monday February 12, 2018 @02:01PM (#16361001)
Amazon is reportedly looking into building their own. Nvidia not only added Tensor cores to the Volta series (impressively quickly, might I add), but they're also creating the NVidia GPU Cloud. Intel has been acquiring DNN hardware startups left and right (Nervana, Movidius, MobilEye) and trying to roll those into their production series.

The hardest part in DNNs is the model and data. That's basically platform-independent. My students mix and match TensorFlow and Caffe, for example, on several different models.

The next part is getting the model implemented in a framework (TensorFlow? Caffe? MXNet? PyTorch?). That's work to change, particularly if you're in a production environment. But it's not the same amount of work as collecting data and building a model.

The final part is running training - CPUs, GPUs, TPUs, etc. This is really fungible. The platform-specific optimizations are relatively small here.

Looking at it from a customer perspective:

  - Can a trained model be exported (weights included) for use on another platform?  (yes)
  - Can the code written for training be used on the customer's own hardware?  (yes, absent any small tweaks needed for TPU, but they're *small*).
  - Might the customer not want to leave because of ease-of-use, particularly at scale, or performance, or total cost of ownership?  (yes, and I think that's what the sales pitch is).
(disclaimer: I worked on part of the Cloud TPU stuff. I'm funded academically by Intel. I have friends at NVidia and own a lot of their GPUs. I love everyone. :)
Re:Cloud TPUs in Beta by puzzle on Monday February 12, 2018 @01:39PM (#16360793)
Why are proprietary drivers a blocker? As long as you expose the same gRPC interface, your customers don't need to know what happens behind the scenes. You could have an FPGA or a Beowulf cluster of Raspberry Pis hiding.
Re:Cloud TPUs in Beta by whataretensors on Monday February 12, 2018 @11:39PM (#16364809)
I should clarify. I like all the individual pieces(hardware, cloud services, grpc interface) I just wish you could opt into them independently.