Re:How to Train Large Models on Many GPUs?
by dauertewigkeit on Saturday February 11, 2023 @12:46PM
(#34754465)
I tried that as well, but maybe I did not use it correctly.
I did not see the full sharding that I was hoping for.
I only saw results similiar to FSDP.
[ Parent
| on HN ]
Re:How to Train Large Models on Many GPUs?
by cma on Saturday February 11, 2023 @04:35PM
(#34756617)
How about flexflow?
https://huggingface.co/transformers/v4.9.2/parallelism.html#...