News for hackers. Stuff that matters.
Re:You can now train a 70B language model at home by jph00 on Friday March 08, 2024 @09:11PM (#39648825)
Yes, you can run FSDP/QLoRA over multi-node. There's a slurm script in the repo showing how to do it.

There are no comments on this comment yet.