Re:Google Cloud TPU Multislice Training
by aschleck on Friday November 10, 2023 @03:59PM
(#38224446)
1979 16 bit flops on an H100 is with sparsity. See footnote 2 on https://www.nvidia.com/en-us/data-center/h100/. You should be halving it for non-sparse flops.
[ Parent
| on HN ]
Re:Google Cloud TPU Multislice Training
by YetAnotherNick on Saturday November 11, 2023 @03:11AM
(#38228450)
GP is correct. With sparsity it is 3958. 1979 Tflop/s is without sparsity.