Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on How to Train Large Models on Many GPUs?. Parent comment
Supports data, tensor, pipeline, sequence parallelisms, activation checkpointing, distributed optimizers, fused kernels and more.
There are no comments on this comment yet.
Supports data, tensor, pipeline, sequence parallelisms, activation checkpointing, distributed optimizers, fused kernels and more.