News for hackers. Stuff that matters.
Re:Introducing the TensorFlow Research Cloud by aub3bhat on Wednesday May 17, 2017 @04:00PM (#14362268)
For comparison, at 180 teraflops per TPU (assumption: at fp32) this is equivalent to offering a cluster with 15,000 Titax Xp GPUs (1200$ per GPU = 18 Million $ worth of compute power).

If TPU teraflops are reported at fp16, then the number would be half.

Titan X offers 12 TFLOPs per GPU https://blogs.nvidia.com/blog/2017/04/06/titan-xp/

Re:Introducing the TensorFlow Research Cloud by fizixer on Wednesday May 17, 2017 @05:00PM (#14362744)
180 = 12 x 15

Where are you getting the extra 3 zeros from?

(Also 'up to 180 TFLOPs' is a bit misleading. As I read in some benchmarking posts by Google as well as by nVidida, a TPU is much faster than a GPU for doing inference, but they haven't released any data on training performance, the real bottleneck IMO).

Re:Introducing the TensorFlow Research Cloud by superfx on Wednesday May 17, 2017 @05:18PM (#14362870)
Also, I suspect the more relevant number is the "Tensor" ops that Nvidia recently reported for Volta at 120 TFLOPS [1], which would be relative to Google's TPU a "mere" 1.5x speed up. Of course, this is pure speculation on my part.

1. https://devblogs.nvidia.com/parallelforall/inside-volta/

Re:Introducing the TensorFlow Research Cloud by asdf_ on Wednesday May 17, 2017 @06:21PM (#14363337)
Google's 180 teraflop number is for a module, which contains 4 TPU chips. Meaning each TPU is 45 teraflops. See: https://arstechnica.com/information-technology/2017/05/googl...
Re:Introducing the TensorFlow Research Cloud by jlebar on Wednesday May 17, 2017 @09:15PM (#14364188)
I replied to this comment elsewhere in the thread [1], but to reply here as well:

A GPU is also comprised of multiple chips (RAM, etc). I don't think "performance per discrete piece of silicon" is an interesting metric.

[1] https://news.ycombinator.com/item?id=14364169

Re:Introducing the TensorFlow Research Cloud by asdf_ on Wednesday May 17, 2017 @10:25PM (#14364432)
A single V100 board is vastly smaller than a TPU module with 4 TPU chips.

The closest comparison in terms of size would be 1 Volta DGX-1 (8x V100s) compared to 2 TPU modules (8x TPU2 chips).

Volta DGX-1: https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent...

TPU Module: https://storage.googleapis.com/gweb-uniblog-publish-prod/ima...

And for completeness, this is the size of a single V100: https://cdn.arstechnica.net/wp-content/uploads/2017/05/34446...

You can see that 8x V100s are still more computationally dense than 8x TPU2 chips. Density is a very important factor in datacenter design.

Re:Introducing the TensorFlow Research Cloud by dgacmu on Thursday May 18, 2017 @09:17AM (#14366914)
From my read of that Volta picture (here's a close-up glamour shot of the V100 module: https://devblogs.nvidia.com/parallelforall/wp-content/upload... ), that's just the chip module itself. Note how none of the interconnect hardware is present? The V100 supports NVLink just like the P100. When you actually assemble that chip module onto a board, it looks more like this:

http://hothardware.com/ContentImages/NewsItem/37034/content/...

(That's a P100 server, obviously, but it's the same kind of design. The P100 module looks a lot like the V100 module.)

Re:Introducing the TensorFlow Research Cloud by asdf_ on Thursday May 18, 2017 @09:40AM (#14367078)
The interconnect hardware is present on the dgx-1 board.

The point of my previous comment is that it doesn't make sense to compare an entire TPU module (containing 4x TPUs) to a V100 board.

The closest comparison to a TPU module (with 4x TPUs) would be a dgx-1 board, which contains the nvlink bus that you mention, but also contains 8x V100 boards, hence why in my previous post I said you should compare the compute performance of a Volta DGX-1 (8x V100s) to 2 TPU modules (8x TPUs).

At the end of the day, it is simple, in a given area of space, you can get more compute performance from provisioning that area with V100s (in the form of using dgx-1s) versus provisioning it with TPUs (in the form of using TPU modules).

Re:Introducing the TensorFlow Research Cloud by jlebar on Thursday May 18, 2017 @12:15PM (#14368474)
> Density is a very important factor in datacenter design.

That's true if you're putting the chips in your own data center, because density affects TCO.

But assuming that Google does not sell TPU hardware, this isn't an important metric to anyone other than Google. The question that is important is, what does the shape of the curve plotting "dollars spent" versus "time spent waiting for my model to train" look like?

The shape of that curve is affected by TCO, and TCO is certainly affected by density. But there's a lot more to it than that.

Re:Introducing the TensorFlow Research Cloud by asdf_ on Thursday May 18, 2017 @12:25PM (#14368559)
Sure, no argument there. For Google, producing and using TPUs, is probably cheaper than buying nvidia hardware.
Re:Introducing the TensorFlow Research Cloud by slizard on Wednesday May 17, 2017 @05:33PM (#14363008)
Might well be at 10x less power -- or more.

Might also be TPU gen2 given the "64 GB of ultra-high-bandwidth memory" was not something IIRC the recent TPU paper was talking about.

And this is here now, while Volta is paper-launched and will be very scarce until late 2017/2018.

Re:Introducing the TensorFlow Research Cloud by dgacmu on Thursday May 18, 2017 @09:08AM (#14366853)
The benchmarking posts were about the previous-generation TPUs, which are detailed in an ISCA paper this year: https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk...

The previous TPU was aimed at inference. The Cloud TPUs do training. But you're right - I haven't seen any publications about training performance yet. But they'll be available in cloud (currently in alpha), and once they go GA, I'd expect to see a raft of benchmarks by third parties. I can't wait. :)

Re:Introducing the TensorFlow Research Cloud by aub3bhat on Wednesday May 17, 2017 @06:40PM (#14363435)
The cluster has 1000 TPUs
Re:Introducing the TensorFlow Research Cloud by fizixer on Wednesday May 17, 2017 @07:09PM (#14363588)
Well, then MS Azure, Amazon AWS have their own clusters offering way more than $18 million worth of compute power (I'm sure they also provide it for research purposes worth in the millions).

But not to a single client.

Similarly, Google is not offering a 1000 TPU cluster to each client.

Your metric is strange to say the least.

Re:Introducing the TensorFlow Research Cloud by aub3bhat on Wednesday May 17, 2017 @07:22PM (#14363661)
They are offering a dedicated 1000 TPU cluster for researchers, at least that's how I read it.
Re:Introducing the TensorFlow Research Cloud by vazamb on Thursday May 18, 2017 @05:22AM (#14365865)
AFAIK in the original TPU announcement they said the cluster would be shared between all participants of the program.
Re:Introducing the TensorFlow Research Cloud by asdf_ on Wednesday May 17, 2017 @06:17PM (#14363318)
It is actually 45 teraflops per TPU2 chip and 180 teraflops for a module (4x chips): https://arstechnica.com/information-technology/2017/05/googl...

V100 is 120 teraflops of tensor ops per chip: https://arstechnica.com/gadgets/2017/05/nvidia-tesla-v100-gp...

Re:Introducing the TensorFlow Research Cloud by jlebar on Wednesday May 17, 2017 @09:12PM (#14364169)
A GPU is also comprised of multiple chips (RAM, etc). I don't think "performance per discrete piece of silicon" is an interesting metric.
Re:Introducing the TensorFlow Research Cloud by asdf_ on Wednesday May 17, 2017 @10:27PM (#14364441)
A single V100 board is vastly smaller than a TPU module with 4 TPU chips.

The closest comparison in terms of size would be 1 Volta DGX-1 (8x V100s) compared to 2 TPU modules (8x TPU2 chips).

Volta DGX-1: https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent...

TPU Module: https://storage.googleapis.com/gweb-uniblog-publish-prod/ima...

And for completeness, this is the size of a single V100: https://cdn.arstechnica.net/wp-content/uploads/2017/05/34446...

You can see that 8x V100s are still more computationally dense than 8x TPU2 chips. Density is a very important factor in datacenter design.

Re:Introducing the TensorFlow Research Cloud by arcanus on Wednesday May 17, 2017 @06:03PM (#14363232)
There is virtually no way this would be fp32. If it was, they could sell these devices for many more tasks than just machine learning.
Re:Introducing the TensorFlow Research Cloud by qeternity on Wednesday May 17, 2017 @04:03PM (#14362300)
Not including power, motherboard, processor, ram, etc...