News for hackers. Stuff that matters.
Re:Tesla bulks up its GPU-powered AI super by algo_trader on Thursday August 18, 2022 @07:39AM (#32508130)
> 7,360-GPU cluster would be capable of about 100 .. petaflops

How is an A100 worth "only" 17 tera-flops?

There are multiple home gpus and even mobile chips rated at tera-flops. And the A100 really is a gigantic chip.

I realize all about bandwidth, and full vs half precision, and inflated marketing vs actual benchmark.

But can a 10-node-iphone-cluster really match an A100 for pure flops?!

Re:Tesla bulks up its GPU-powered AI super by nodja on Thursday August 18, 2022 @08:41AM (#32508784)
> But can a 10-node-iphone-cluster really match an A100 for pure flops?!

Only if the data is local, small, easily synchronizable, doesn't require fp32 precision and a dozen other caveats.

Can a 10 iphone cluster match an A100 in calculating digits of pi? Yes. Can it do backpropagation in large machine learning models? The gradients won't even fit.

Re:Tesla bulks up its GPU-powered AI super by redox99 on Thursday August 18, 2022 @08:48AM (#32508866)
Because its hardware is not optimized for FP32. If you're doing FP32 then a 3090 is literally faster.

It's optimized for TF32, INT8, bfloat16, and other similar data types that the tensor cores like.

Re:Tesla bulks up its GPU-powered AI super by make3 on Thursday August 18, 2022 @01:57PM (#32512842)
the number is for fp64. gpus most definitely are optimized for fp32
Re:Tesla bulks up its GPU-powered AI super by calaphos on Thursday August 18, 2022 @09:49AM (#32509543)
The number is for FP64 double precision terraflops. For vector operations, not matrix operations as well.

Double precision is important for a lot of simulation codes because of numerical issues. For Deep learning applications it's basically unused, single precision or half precision formats dominate training. The same numerical issues don't show up, so this is a lot more efficient.

Additionally a lot of the 'AI' flops are achieved through matrix cores, fitting Deep learning the workload well. This brings an A100 from only 17 to ~200 tflops.

> But can a 10-node-iphone-cluster really match an A100 for pure flops?!

The iphone number is probably fp32 if not lower precision. However the combined SoCs of 10 iphones have tripple the transistor count and a larger chip area among them. So the comparison is not totally off.

Re:Tesla bulks up its GPU-powered AI super by tuankiet65 on Thursday August 18, 2022 @08:51AM (#32508903)
That 17 tera flops is in double precision (FP64) math as benchmarked by TOP500.
Re:Tesla bulks up its GPU-powered AI super by _kbh_ on Thursday August 18, 2022 @08:08AM (#32508438)
The A100 is not optimised for pure ieee-754 FP throughput it seems but for more specialised forms of FP and bandwidth.

https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent...